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PREFACE 


This book contains introductory material on numerical methods of 
solving engineering problems by digital computers. The book is written 
for engineers and scientists who desire to achieve a working knowledge 
of digital computer calculations and programming methods for subse- 
quent application in their field of practice. For this reason, the book 
covers basic digital computer methods which can be applied to all 
fields of engineering, rather than specific applications. The level of 
the contents is such as to make the book particularly suited for self- 
study. Each chapter can be taken independently for the study of a 
specific subject or a specific application. 

The book can be used as a junior-senior level text for a one semes- 
ter course in digital methods given to engineering students from vari- 
ous branches of engineering or as a reference book to supplement the 
numerical computation portion of general applied mathematics courses. 

vil 


viii Preface 


It is assumed that the readers have had two years of engineering 
mathematics. No knowledge of computer programming is assumed on 
the part of the reader. Digital computer programs are given for a number 
of numerical examples to familiarize the reader with actual computer 
programming logic. These programs are written in FORTRAN (used by 
general purpose computers) and BASIC (used by time-sharing computers) 
programming languages. These programming languages are so close to 
actual algebraic statements that, with the explanatory paragraphs given 
with each program, the reader will have no difficulty following the 
logic of the computer programs. It should be noted that the digital 
computer programs are written primarily for simplicity of logic rather 
than efficiency of computations. 

The following is a general description of the contents of various 
chapters, 

Chapters | and 2 contain discussions and methods of numerical solution 
of determinants, matrices, and linear algebraic equations. Included in 
these chapters are methods of evaluating characteristic values and vectors 
using digital computers. A physical interpretation of characteristic values 
and vectors is given at the end of the second chapter. 

Chapter 3 includes a number of interpolation formulas and methods of 
curve fitting primarily used with statistical data. Polynomial root finding 
methods are also described in this chapter. 

Chapter 4 gives a discussion of time-frequency domain analysis using 
Fourier series and transforms and describes the use of digital computers 
in spectral analysis of time dependent functions. 

Chapters 5, 6, and 7 start by describing the basic relations between 
the differential and difference operators and methods of reducing a 
differential equation to a difference equation. Subsequent discussions 
deal with solution of various types of differential equations using differ- 
ence methods and digital computers. An appendix containing elementary 
solutions of typical differential equations is included for readers desiring 
a review of classical methods of solution of differential equations. 

Chapter 8 describes the numerical methods of solution of partial 
differential equations. Specifically, the numerical solutions of Laplace’s 
equation, the heat equation, and the wave equation are presented. 

Chapters 9 and 10 give a discussion of the elements of linear and 
dynamic programming. On the assumption that the majority of readers 
are not familiar with the subjects of these chapters, these chapters are 
written at a somewhat lower level than the rest of the text. 


Most of the material described above is taken from lecture notes of 
applied mathematics courses offered at UCLA over the past several 
years. In addition, a number of computational methods currently used 
in industry are included. For this reason the authors wish to acknowledge 
the valuable suggestions and comments of their students and their col- 
leagues during the preparation of this work. The editorial assistance of 
Mrs. Rosalie Kenniston of Hughes Aircraft Company in the preparation 
of the original manuscript is also gratefully acknowledged. 


S. A. Hovanessian 
Louis A. Pipes 
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7 DETERMINANTS, 
MATRICES, AND LINEAR 
SIMULTANEOUS 
EQUATIONS 


The theory of determinants is intimately connected with that of 
matrices and with the solution of simultaneous linear algebraic equations. 
A great deal of the terminology and some of the operations involved 
in matrix algebra are based on a knowledge of elementary deter- 
minant theory. In this chapter we start with a brief discussion of the 
elements of the theory of determinants. This elementary discussion 
should provide an adequate review for the reader who is primarily 
interested in matrix algebra and matrix calculus from a standpoint 
of applications to engineering and physics. The rest of the chapter 
is devoted to matrices and numerical solutions of matrix equations 
and linear simultaneous equations. 
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DETERMINANTS AND SIMULTANEOUS 
EQUATIONS 


It is sometimes convenient to express the combination of terms (ad-bc) 
in the following form: 


a c 
= ad-be (1.1) 


D is called a determinant of the second order. The letters a, b, c, d, are 
called the elements of the determinant; (ad-be) is called its expansion. 
The first row of D consists of the elements a and c. The second row of 
D consists of the elements b and d. The elements a and b comprise the 
first column and the elements c and d the second column. 

A determinant of the third order has three rows and three columns; 
a determinant of the order n has n rows and n columns. The transpose 
of a determinant D of any order will be denoted by D’; it is the deter- 
minant formed by interchanging the rows and columns of D. 

It is therefore evident that if 


Des 


Determinants of the nth Order 


An nth order determinant A is a square array of n? quantities ai; 
called the elements of the determinant written in the form 


ag Wha bas D5 Ain 
a, a. (cag: 

aes! 22 23 2n (1.3) 
ay a2 a3 Gan 


Consider the following definitions of quantities with respect to the 
above determinant. 

Minors. If in the determinant A of Eq. (1.3) we delete the ith row 
and the jth column and form a determinant from all the remaining 
elements, we shall have a new determinant of (n — 1) rows and columns. 
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This new determinant is defined to be the minor of the element a; ;. The 
minor of the element a;; will be denoted by M;;- 

Cofactors. The cofactor of an element of a determinant a;,; is the 
minor M;; of that element with a sign attached to it. The sign is deter- 
mined by the numbers i and j which fix the position of a;; in the deter- 
minant A. The sign is chosen by the equation 


C;, = -Di*M,, (1.4) 


where Ci; is the cofactor of the element a;; and M,, is the minor of the 
element a if 

In the general nth order case if C;; is the cofactor of any element a;; in 
the determinant A, the determinant can be expressed as the sum of the 
elements in any one row or any one column each multiplied by its co- 
factor. That is, 


D= Saha (any row i) Ci:5) 


i 


D = Da, :6,5 (any column j) (1.6) 


i 


By repeating the expansion represented by Eq. (1.5) or (1.6), a deter- 
minant of large order n can be expressed in terms of determinants of 
steadily smaller order and hence can be evaluated. 

It is easy to demonstrate that, in general, 


n n 
Dat t= ek 

GnCk = @:,0,, = (1.7) 
» oii » au ss : PER 


i=1 i= 


It should be noted that the evaluation of determinants in terms of 
cofactors as given by Eqs. (1.5) and (1.6) involves excessive numbers 
of arithmetical operations. It can be shown that by using this method, 
the number of multiplications can be approximated by n! where n is the 
order of the determinant. For a determinant of order 10 this will result 
in 10! or 3.6 x 10° multiplications. In succeeding sections, methods 
using the fundamental properties of determinants will be given where 
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the number of multiplications is reduced to n?/3, or about 300 multipli- 
cations for evaluating a 10th order determinant. 


Fundamental Properties of Determinants 


From the basic formulas (1.5) and (1.6), the following properties of 
determinants can be deduced: 

1. If all the elements of a row or of a column are zero, the determi- 
nant is equal to zero. This may be seen by expanding in terms of the 
elements of that row or column in which the elements are equal to zero. 

2. If all elements but one in a row or column are zero, the determinant 
is equal to the product of that element and its cofactor. 

3. The value of a determinant is not altered when the rows are 
changed to columns and the columns to rows. In other words, the 
determinant and its transpose are equal. 

4. The interchange of any two columns or two rows of a determinant 
changes the sign of the determinant. 

5. If two rows or two columns of a determinant are identical, the 
determinant is equal to zero. 

6. If all the elements in any row or any column are multiplied by any 
factor, the determinant is multiplied by that factor. 

7. If each element in any column or any row of a determinant is 
expressed as the sum of two quantities, the determinant can be expressed 
as the sum of two determinants of the same order. 

8. It is possible, without changing the value of a determinant, to 
multiply the elements of any row (or any column) by the same constant 
and add the products to any other row (or column). 


Use of the Fundamental Rules in Evaluating 
a Determinant 


The fundamental rules enumerated above may be used in the numerical 
evaluation of determinants. For example, let it be required to evaluate 
the determinant 


2k 6s a 
Lie day 16% 43 
ee (1.8) 
Seas Rane ar 
SUF Wi Snerh 
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This determinant will be greatly simplified if we use the above properties 
to make all its elements except one in some row or column equal to 
zero. The presence of the factor —1 in the second column suggests that 
we add four times the first row to the second, then add two times the 
first row to the third row, and finally add the first row to the fourth row. 
By principle 8 these operations do not change the value of the determi- 
nant. Hence we have 


Diemer AL 
9 0 2 aa oe 
7 
Dim = 8 at '6 O2) 
Soe iar 6 
5. 7 °6 
OO tT 


We now subtract the elements of the last column from the first column 
of the third-order determinant of Eq. (1.9) and thus obtain 


2 2 7 

1.10 

Dect ye ih 16 ( : 
-1 7 6 


We now add two times the third row to the first row and two times the 
third row to the second row and thus obtain 


0 40 19 ‘ 
40 19 
D=]|0 31 18] = - Oe 
31 18 
Sh aioe aa 


Now we may subtract the second row from the first row and obtain 


= -(162 - 31) = -131 (1.12) 


The procedure described above is much shorter than a direct appli- 
cation of Eq. (1.5) or (1.6). 
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Chio Method for Evaluating Determinants 


The above procedure can be systematically applied to reduce the order 
of large determinants for numerical evaluation. In matrix algebra this 
procedure is referred to as the method of Chid. Consider the 4 x 4 
determinant 


it Me Aig. ie 
a, a a, a. 
21 no Sag Sag 
D= (1.13) 
Pao sa. ehSae coo 


Tate Rage tag (Say 


The idea of this method is to reduce the order of the determinant from 
4 to 3 and from 3 to 2, and so on. This results in a 1 x 1 determinant 
which will give the actual value of the original determinant. To describe 
this, we proceed as follows. Make any element of the determinant, say 
element (1, 1), equal to unity by dividing the first row through by ay). 
This results in 

5° Sis 1g 


1 — 
ag es Toe G1 
D = a),}%1 %2 43 44 (1.14) 
931 “839: “Gag ntiag4 
41 %2 3 444 
Denoting a}, = a,9/a, 1, @j3 = 913/411, @j4 = 4,4/a,, and multiplying 
the first row by a, and subtracting from the second row, then multi- 
plying the first row by ag, and subtracting from the third row and, 


finally, multiplying the first row by a4, and subtracting from the fourth 
Tow, we have 


1 a9 ai3 4 
0 agg — A914} Gg9 -~ 421 G}3 agg ~ G91 O44 (1.15) 


D’=a 
1} 


939 — 4314} 933 — 431413 434 — 31244 


0 G9 - 441449 M43 - 441413 O44 - 441 O44 
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The value of this determinant now becomes 


a, 2 413 
D’ = ay,(-I)'*")a3, apa (1.16) 


43; 432 933 


where (—1)!*! is the sign of the element (1, 1) and double primes are 
used to denote elements 


ay, = Agg — 491 4} 45 = Agq — Ap, 4}9,--- 
11 22 21°12* 12 22 2113) 
(1.17) 


51 = G39 - 43149, 932 = 433 ~ 931 413+ --- 


and so on. Thus Eq. (1.16) is a determinant of order 3 x 3. Repeating 
the above operation, this determinant will reduce to a 2 x 2 and subse- 
quently to a 1 x 1 with multipliers a, , aj, a7,-... Thus, the value of the 
original 4 x 4 determinant can be obtained. 

In cases where any of pivotal elements— aj, a},, @[}, ---in the above 
derivation—become zero, the appropriate columns or rows should be 
interchanged to obtain a nonzero pivotal element. The process can then 
be continued. If an element which is already unity exists in a certain 
row, that element can be taken as the pivotal element, thereby reducing 
the number of divisions required to obtain a unity pivotal element. This, 
however, in a digital computer program will complicate the logic and will 
require additional logical statements. 

The total number of arithmetical operations* required in evaluating 
determinants by the method of Chio can be shown to be 


(n — 1)(2n? — n + 6) 
6 


n(n — 1) 
2 


multiplications 
(1.18) 


divisions 


*Multiplications and divisions rather than additions and subtractions are used as a 
basis for calculating the required digital computer computation time, since multipli- 
cations and divisions take far more (by a factor of 5 to 10) computation time in 
digital computers. 
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where n is the order of the determinant to be evaluated. Fora 10x 10 
determinant the above relations give, approximately, 275 multiplications 
and 45 divisions. Assuming that each operation will take 50 micro- 
seconds on a digital computer, the total computation time will be 320 x 
50 x 10° = 16 milliseconds. 


FORTRAN Program for Determinant Evaluation 


The logic of evaluating determinants by the method of Chid, as dis- 
cussed above, can be programmed for digital computer solution. Such a 
program, written in FORTRAN language, is given below. 


CSAH1 DETERMINANT EVALUATION PROGRAM 
10 DIMENSION S(100,100) 
20 DATA N/3/ 
30. = DATA ((S(1,J),J=1,3),1=1,3)/1.,2.,3.,3.,4.,6.,2.,1.,1./ 


40 L=1 
50 K=2 
60 XM1=1. 


70  XM=S(L,L) 

80 DO95J=L,N 

90 = S(L,J)=S(L,J)/XM 

95 CONTINUE 
100 DO 1401=K,N 
110 = X=S(I,L) 
120 DO 140 J=L,N 
130 — S(I,J)=S(1,J)-S(LJ) *X 
140 CONTINUE 
150 L=L+1 
160 K=K+1 
170 XM1=XM1*XM 
180 IF (L-N)70,190,190 
190 XM1=XM1*S(N,N) 
200 WRITE (2,210) XM1 
210 FORMAT (22H VALUE OF DETERMINANT= 1PE10.3) 
220 STOP 
230 END 


The computer solution obtained is 


VALUE OF DETERMINANT = 1.000E00 
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In order to completely understand the above program the reader should 
consult the FORTRAN reference manual listed as [1]. The logic of the 
program, however, can be explained by considering the evaluation of the 
following 3 x 3 determinant used in the program. It is required to 
evaluate the determinant 


LS 20e33. 
yee er (1.19) 
2. i 


Statement 10 of the program will reserve computer memory space for a 
100 x 100 determinant. Statements 20 and 30 will read into the com- 
puter memory the order of the determinant N = 3 and the elements of 
the determinant S(/,J), starting from the first row and proceeding to 
the second and the third rows. Statements 40 through 60 will set indices 
and the value of XM 1 = 1.0 to be used later in the program. Statement 
70 will set XM equal to the value of element (1, 1) or S(1, 1) of the 
determinant or in this case S(1, 1) = XM = 1.0. The loop consisting of 
statements 80 through 95 will divide the elements of the first row by 
XM. In this case, since XM is equal to 1, determinant of Eq. (1.19) will 
be unchanged. The double loop consisting of statements 100 through 
140 and 120 through 140 will multiply the first row by element S(2, 1) 
and subtract the resulting first row from the second row and will repeat 
this process with the third row. At the end of this operation, the 
resulting determinant will be 


Tee ak 
aw (1.20) 
0)=8) =6 


Statements 150 and 160 set in indices L= L+ 1 and K =K + 1, resulting 
in L = 2 and K = 3. The value of XM1 is equal to previous XM 1 times 
XM, or at this point XM1 = 1.0. Thus, at statement 170 the determinant 
is reduced to a second-order determinant 


-2 -3 


(1)X (1.21) 
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Statement 180 checks to see if the value of L is smaller than the order of 
the determinant, in which case the above operation starting with state- 
ment 70 is repeated. The second application of logic described by 
statements 70 through 170 will result in 


Yaa Boat 


(1) (2) (1.22) 
0 -5 


This time, since N will be equal to L at statement 180, the logic will go 
to statement 190. This will set 


XM1 = (1)(-2) (-.5) = 1 (1.23) 


which is the value of the determinant as printed in the computer solution. 
Note that in the above program the 3 x 3 determinant was kept in 
computer storage while its elements were changed and operated upon 
without manipulating the location of unused elements. 


The Solution of Linear Equations; 
Cramer’s Rule 


A general system of n simultaneous linear equations can be written in 
the form 


41% + QygQXo + ++. + y,X, = hy 
21%, + Gp9X%p + --- + GanX, = he (1.24) 
G,1%1 + GnpXo +--+ + An_X, = hep 


The solution of the system (1.24) can be shown to be given by 


D 


r 


Be SL 2S ons 
D (1.25) 


r 
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where D is the determinant of the coefficients of Eqs. (1.24) 


(1.26) 


The determinant D, is obtained by replacing the rth column of D by 
(ky, Rg, kg, --+, k,,). The result (1.25) is called “Cramer’s Rule” in the 
mathematical literature. It presents an elegant but laborious method 
for solving sets of simultaneous equations. 

From Eq. (1.18) for large values of n, high-order determinants, the 
number of multiplications can be approximated by n®/3 and divisions 
by n2/2. Assuming the number of divisions to be negligible as compared 
to the number of multiplications, the number of arithmetical operations 
in evaluating nth order determinants by the method of Chid can be set 
at n3/3. Using Cramer’s rule in solving n simultaneous equations in n 
unknowns, n + 1 determinants should be evaluated. Using Chid’s 
method for the evaluation of these determinants will result in the total 
number of arithmetical operations (n3/3) (n + 1) = n4/3. In succeeding 
sections it will be shown that Gauss’ method of elimination requires 
only n3/3 multiplications in solving n simultaneous equations in n un- 
knowns. For this reason Cramer’s rule should be taken more for its 
historical and academic value rather than for its value in practical 
application. 

The Regular Case. If the determinant of the coefficients D of the 
equations does not vanish so that D # 0, the set of equations is said to 
be regular. In this case they have the unique solution (1.25). 

The Singular Case. If D = 0, the set of equations is said to be singular. 
A singular set of equations may or may not be consistent. The question 
of consistency will not be considered here. However, it may be pointed 
out that in simple cases the question of consistency can be decided by 
trial. 


Homogeneous Equations in ” Unknowns 


If all the numbers hy, ky, ky, ---, k, in Eqs. (1.24) are zero, the set of 
equations is said to be a set of homogeneous equations. It can be proven 
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that the necessary and sufficient condition that a set of homogeneous 
equations should have nonzero solutions (x,, x9, xg, ..., x, not all zero) 
is that the determinant of the coefficients D should vanish, or D = 0. 

An important relation between the ratios of the unknowns in the 
case of nontrivial solutions can be established in the following heuristic 
manner. Let it be assumed, for the moment, that the determinant D is 
not zero and that only one of the k’s in Eqs. (1.24), say k,, is different 
from zero. In this case by Cramer’s rule we would have 


ae d (1.27) 


x, 
ee fore ihe sl ONE (1.28) 
Xs 


The result (1.28) is independent of the values of both k; and the 
determinant D. It may be inferred, therefore, that Eq. (1.28) holds also 
when both k; and D are zero. The correctness of this conclusion can be 
demonstrated in a rigorous fashion. It is therefore interesting to note 
that when the set of homogeneous equations has nontrivial solutions, 
the ratios of the unknowns are fixed by Eq. (1.28) in terms of the 
ratios of cofactors of the determinant D. 


MATRIX ALGEBRA, LINEAR 
SIMULTANEOUS EQUATIONS 


Matrices provide a compact and flexible notation that is particularly 
useful in studying linear transformations, and present an organized 
method for the solution of systems of linear algebraic and linear differ- 
ential equations. Since many engineering problems are formulated in 
terms of linear algebraic or linear differential equations, it is natural that 
their solution will be expedited by the use of matrix algebra and matrix 
calculus. Matrix algebra has been found so useful in pure and applied 
mathematics that it has been termed ‘‘the arithmetic of higher mathe- 
matics.” 
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Definition and Notation 


A table of n by m quantities, called elements, arranged in a rectangular 
array of n rows and m columns is called a matrix with n rows and m 
columns. If a;; is the element of the ith row and the jth column, then 
the matrix A can be written in the following manner: 


ie Go, [9 [29 .--+ Fam (1.29) 


The matrix A of Eq. (1.29) is said to be a matrix of order (n,m). If the 
number n of rows is equal to the number m of columns, then the matrix 
A is called a square matrix. 

Besides square matrices, two other general types of matrices frequently 
occur. One is the row matrix R: 


R= fry ee Tl (1.30) 


The other is the column matrix C 


ae (1.31) 


Elementary Operations on Matrices 


Transposition. It is possible to interchange the rows and the columns 
of the matrix A of Eq. (1.29) to obtain a new matrix A’ of the form 


Bip Pay “Sar G1 

7 ae lice eda tat a2 
Cas, eee eer. Mee. (1.32) 

Bim ~ 92m 93m ++ Sam 
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The matrix A’ is called the transpose of A. The transpose of the row 
matrix Eq. (1.30), R’, is a column, and the transpose of the column 
Eq. (1.31), C’, is a row matrix. 

Addition of Matrices. Let A and B be two matrices of the same order, 
that is, they have the same number of rows and columns. Let 


A =[a,], B= [bj] (1.33) 


ij 
Then by the sum A + B of the matrices A and B we shall mean the 
uniquely obtainable matrix C whose elements c;; are obtained by the 
equation 


ea LEQ Nae 
Ces Os + De » C=A+B (1.34) 
(ieee br ee gee 


In other words, to add two matrices of the same order, we calculate 
the matrix whose elements are precisely the numerical sum of the 
corresponding elements of the two given matrices. The addition of two 
matrices of different orders is not defined. 

Equality of Matrices. To complete the preliminary definitions, we 
must make clear what is meant when two matrices are said to be equal. 
Two matrices A and B are defined to be equal provided that they have 
the same order and that their corresponding elements are equal. That is, 


AaB, if) es. 0.y (1.35) 


From the above definitions of the sum of two matrices and the equality 
of two matrices it follows that addition of matrices is associative, that is, 


A+(B+C) = (A+B)+C =A+B+C (1.36) 


Before proceeding with the definition of multiplication of matrices, a 
word or two must be said about two very important special square 
matrices. One is the zero matrix—a matrix all of whose elements are 
zero, for example, 
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Cue 0) Ol 
0=]0 O O- OJ, a zero matrix of order (3, 4) (1.37) 
0; ee 


Another matrix of great importance is the unit matrix. This is a square 
matrix of any order whose principal diagonal elements are unity and has 
zeros for all its other elements; it is denoted usually by the letter I. 

For example, the unit matrix of the fourth order is 


ty Ole 

A eR a (1.38) 
0: =OnmienO 
eeu 


Multiplication of Matrices 

Multiplication of a Matrix by a Scalar. By definition, the multiplica- 
tion of a matrix A by an ordinary number or scalar k is effected by 
multiplying each element of the matrix A by the number k, thus obtain- 
ing a new matrix whose elements are ka; ;. That is, by definition, 


B = kA where b.. = ka.. (1.39) 


For example, 


2 #4 6 2 
>f ] 2 ne 
fg et! 9 3 


The Multiplication of a Matrix by Another Matrix. The definition 
of multiplication of one matrix by another has been chosen in order to 
facilitate operations involving linear transformations by the use of matrix 
algebra. 

The rule of multiplication is such that two matrices A and B can be 
multiplied together in the order AB only when the number of columns 
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of A is equal to the number of rows of B. That is, if A is a matrix of 
order (m,n) and B is a matrix of order (p,q), the product AB is not 
defined unless n = p. If n = p, the matrices A and B are said to be con- 
formable, In matrix algebra only conformable matrices can be multiplied 
together. 

The product of a matrix A of order (m, p) by a matrix B of order (p, n) 
is defined to be the matrix C given by 


C = AB = [e;,] (1.41) 
where the elements Ci; of the matrix C are obtained by the equation 


k=d 
ey = ai, b,j (1.42) 
Pes 


It can be seen that the product matrix C is a matrix of order (m,m). The 
fact that a matrix of order (m, p), when multiplied by a matrix of order 
(p,n), results in a matrix of order (m,n) may be expressed symbolically 
in the following manner: 


(m, p)(p,n) = (m,n) (1.43) 
This symbolic equation is useful in determining the order of the 


resulting matrix when several matrices are multiplied together. As an 
example of the definition of the rule of multiplication Eq. (1.42), let 


%, 2 448 by, ae 
A= |40)  ay3 dg] BB = | bg, gg (1.44) 
93, 432 3g bg, bgp 


In this case we have matrices of orders (3, 3) and (3, 2). They are 
therefore conformable and the product matrix C is of order (3, 2). In 
this case the typical element Ci of the product C = AB is given by Eq. 
(1.42) in the form 


iy = Fi1 Oyj + jg ba; + G53 03; (1.45) 


L eal 
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The elements Eq. (1.45) are now arranged in the rectangular array, 
(a1 yy + Gobo + @y3b51) (ay bya + Ayybog + 413 439) 
C = | (ay, byy + 499691 + 493633)  (4p1 byy + Apg bog + Gg3 b39) 
(a5) bj, + 439091 + 433631) (31 byg + Gg boo + 33 bgp) 
(1.46) 


where C is the product matrix AB. It can be seen that the result Eq. (1.46) 
can be obtained by taking the sum of continued products of the rows of 
A times the corresponding columns of B. 

Properties of Matrix Multiplication. In general, matrix multiplication 
is not commutative, that is, 


AB # BA (1.47) 


This can be seen directly from the definition Eq. (1.42), The matrices, 
Eq. (1.44), are not conformable in the order BA and cannot be multi- 
plied in this order. 

In matrix algebra it is necessary to distinguish between premultiplica- 
tion, as when A is premultiplied by B to give the product BA, and post- 
multiplication, as when A is postmultiplied by B to give the product 
AB. If we have the equality 


AB = BA (1.48) 


then the matrices A and B are said to commute or to be permutable. The 
unit matrix J, it can be noted, commutes with any square matrix of the 
same order. This is, 


IA = Al (1.49) 


provided that / and A are of the same order. 

Continued Products of Matrices. Except for the fact that matrices do 
not commute in multiplication, all the ordinary laws of scalar algebra 
apply to the multiplication of matrices. Of particular importance is the 
associative law of continued products: 


ABC = A(BC) = (AB)C (1.50) 
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The fact that the associative law is satisfied permits one to dispense with 
parentheses and to write ABC without ambiguity. It must be noted, 
however, that the product of a chain of matrices will have meaning only 
if the adjacent matrices of the chain are conformable. 

The Reversal Law of Transposed Products. It can be shown as a con- 
sequence of the definition of the rule of matrix multiplication Eq. (1.42) 
that the transpose of a product of matrices is equal to the product of 
their transposes in reverse order. That is, if we have 


M = AB M = B’A’ (reversed order) (1.51) 


where primes indicate transpose matrices. Similarly, if 
M = ABC then M’ = C’B’A® (1.52) 
This result is called the reversal law of transposed products. 


Partitioned Matrices and Partitioned 
Multiplication 


It is sometimes convenient for purposes of computation to extend the 
use of the fundamental laws of combinations of matrices to the case in 
which a matrix is considered to be constructed from elements that are 
themselves matrices. For example, consider the matrix A 


Ate} Poa: 429 993 | = a. (1.53) 
este. es te Sea) 


The matrix A above has been partitioned by the introduction of a vertical 
and a horizontal line into the four submatrices P,Q, R, and S given by 


a a a R= {a3) 39) 
Peal 12 Q- 13 
Sate Pan 993 S = a33 
In this case, the matrix A has been partitioned in such a manner that 
the diagonal submatrices P and S are square, and the partitioning is 


" 
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diagonally symmetrical. Now let B be another square matrix of the third 
order partitioned similarly to A in the form 


bh, Oye bi3 
P, Q 
B=| 21 22 | bog |-| > “1 (1.54) 
Tekan | areas R, Sy 
bs; bgp | Ogg 


Now it can easily be seen that the sum C = A + B can be expressed in 
terms of submatrices in the form 


AP SrEy ) 
Ce Agee | ee a (1.55) 


MRE R NSS) 


This indicates that when two matrices of the same order are partitioned 
similarly, their sum can be obtained by adding their various submatrices 
as if they were elements. 

Multiplication in Terms of Submatrices. As a consequence of the fun- 
damental rule for the multiplication of matrices, a rectangular matrix B 
can be premultiplied by another rectangular matrix A, provided that the 
two matrices are conformable, that is, when the number of rows of B 
equals the number of columns of A. Now if A and B are both parti- 
tioned into submatrices such that the grouping of columns in A agrees 
with the grouping of rows in B, it can be shown that the product AB 
can be obtained by treating the submatrices as ordinary elements and 
proceeding according to the general rule of multiplication. 

In the case of the matrices A and B discussed above, the partitioning 
is such that the product D = AB can be carried out by treating the 
submatrices of Eqs. (1.53) and (1.54) as if they were ordinary elements, 
thus obtaining 


D = AB 


(PP, + QR,) (PQ, + QS,) 
| 1 1 1 '] (1.56) 


~ -|(RP, + SRy) (RQ, + SS,) 
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Matrix Division; the Inverse Matrix 


Having defined the addition and the multiplication of matrices, we 
now consider an operation that is analogous to the operation of division 
in ordinary scalar algebra. In ordinary algebra, if the product of two 
quantities b and a equals unity, so that 


ba = 1 (1.57) 


we say that b is the reciprocal of a and we write 
Boe a (1.58) 
a 


In matrix algebra the unit matrix | of Eq. (1.38) has properties similar 
to that of the number one in ordinary algebra since Al = JA = Aif A isa 
square matrix of the same order as J. If we now have two square matrices 
A and B of the same order and if they satisfy the equation 


BA =I (1.59) 


we see that Eq. (1.59) is the matrix analog of Eq. (1.57); we then say that 
B is the inverse matrix of A and we write 


Boa A> amd) A-SAv— I (1.60) 


Equation (1.60) is the analog of the scalar Eq. (1.58). 

It can be shown that the necessary and sufficient condition for the 
inverse of matrix A is that the matrix A be nonsingular, that is, |A| # 0. 
In order to find the inverse of matrix A, let us define the adjoint of A 
as the transpose of the matrix formed by replacing the elements of A by 
their cofactors C;;. For a (3 x 3) matrix, this is written as 


adjoint (A) = adj A =|Cj. Cyg Cao (1.61) 
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On the strength of Eq. (1.7) we can write 


(A)(adj A) = D[N (1.62) 


where D(= |A|) is the determinant of matrix A and [I] is the unit matrix. 
From the above equations the inverse matrix can be written 


adj (A) 1 = cay (1.63) 
D (A) 
or 
fol adj A (1.64) 
|A| 


In practice the inverse of matrix A is formed by obtaining the transpose 
of matrix (A’) and subsequently obtaining the matrix of the cofactors 
of A’. Division of the matrix of the cofactors by the value of the 
determinant of A(- |A|) will give the inverse matrix A™. 

Although the above method of matrix inversion represents the classical 
approach, computationally it involves an excessive number of arithmetical 
operations. It has been shown that evaluation of large-order determi- 
nants by the method of Chid involves, approximately, n3/3 multiplica- 
tions, where n is the order of the determinant. Using this as a basis, 
evaluation of the inverse of an n by n matrix by the method of cofactors, 
described above, will involve evaluation of n? cofactors of order (n - 1) 
by (n — 1) and, in addition, the determinant of the matrix, resulting in 
(n2)(n — 1)3/3 + n3/3 multiplications. For large n the above can be 
approximated by n5/3 multiplications. In the following sections it will 
be shown that matrix inversion, for example, by the aigmented matrix 
method, can be accomplished by as few as n> multiplications, a reduction 
by a factor n2/3 from the number of multiplications required by the 
cofactor method of matrix inversion using the method of Chio in 
evaluation of the cofactors. 


Gauss’ Method of Elimination in Solving 
Simultaneous Equations 


This method is probably the simplest approach to the solution of a 
series of simultaneous equations. It consists of eliminating one unknown 
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at a time from the set of equations and proceeding with the remaining 
equations. This, at the end, will result in only one unknown and one 
equation. Working backward from this equation, one can solve for one 
unknown at a time until the solution is complete. Consider the set of 
equations 


411%, + @QX%o + GygX3 +--+ + Gy,xX, = dy 
G91X1 + AggXq + AggXy + +++ + Ay,X, = dy 
SORORITIES Con a eT or 2 (1.65) 
Gj), + GjoXq + GjgX3 + +++ + Gj,X, = 6; 
G1, + AngXq + GygXg + +++ + Gq_X, = db, 


Solving the first equation for x,, we get 


Substituting this value of x, in the second through nth equation, we 
obtain a set of n — 1 equations in n— 1 unknowns x, through x,. This 
set can be written 


AyoXy + ApgXz + +++ AynX,_ = by 
Gigxy + Gigty + +++ Cink, = bj ee 
GaXq + @n3%y + +++ BanX_ = by 


where primes denote the new coefficients. Now, solving the first 
equation for x, and substituting in the second through nth equations, 
we obtain a set of n — 2 equations in n — 2 unknowns x3 through x,,. 
Continuing this operation will result in two equations in two unknowns 

w 


Cn -1,n-1%n-1 + Cn-1,n% = 4n-1 


(1.67) 


Crean 1 Xmail th Cnn *ne= Sn 
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and finally in one equation and one unknown 
Cn n®n Sida (1.68) 


where c’s and d’s represent constants. Thus, Eq. (1.68) will give the 
value of x,. Having x,,, Eq. (1.67) will give the value of x, _ ,. Continu- 
ing this process will result in values of x,,_9, X,-3) +++» %2 and x. 

It can be shown that the number of multiplications and divisions 
required in the above method is given by 


n(n + 1I)(QQn + D 
6 


multiplications 
n divisions 


where n is the number of simultaneous equations to be solved. For large 
n, the above values (neglecting divisions) can be approximated by n3/3, 
a factor of n advantage over the solution by Cramer’s method using 
Chio’s method for the evaluation of the determinants. 


Triangulation Method in the Solution of 
Simultaneous Equations 


The triangulation method is often used in obtaining the solution of a 
set of simultaneous linear equations. This method primarily consists of 
systematically reducing the number of unknowns from each equation, 
and ending up with a single equation and a single unknown. One can 
solve this equation for the unknown and work backward to solve for all 
the rest of the unknowns one at a time. To describe this in equation 
form, consider the set of linear equations 


Q11)%X1 + GyoX%_ + +++ + A,X, = by 
p1X1 + ApgXy + +++ + Ga,Xq_ = bg 
: (1.69) 
GijX1 + GjgXq + +++ + GinX_ = 0; 


GX, + GngXq + +°+ + GnpXy, = b, 
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Divide through the first equation by a,,. Multiply this equation by a9) 
and subtract from the second equation, Multiply the first equation by 
a3, and subtract from the third equation. Follow this procedure up to 
the nth equation. This will result: 


X] + GjQXQ + +++ + ay,x, = by 
A59X%_ + +++ + G9,X, = b9 
: (1.70) 
GigXg t+ +++ + GinX, = dF 
Gngxg t+: + On_X, = 0, 


where primes denote new coefficients. 
Now divide the second equation by a, and subsequently multiply the 
resulting equation by 39, a4), -.. and subtract from the third, fourth, 
- , equations, respectively. This operation will eliminate unknowns x,'s 
from all of the equations except the first and the second equations. 
Continuing this procedure will triangulate the set of equations resulting 
in 


x) + CyoX%q + CygX%q + +++ + Cy),X, = d, 
Xo + CogXg + +++ + ConX, = do 
SS 
ne (1.71) 
pxiye et n-1 
n = 4, 


where c’s and d’s are constants resulting from arithmetical operations. The 
last equation gives the value of x,. This x, can be substituted in the 
equation before last resulting in the value of x,_,. Continuing this 
process, all of the unknowns can be obtained. 

The triangulation method can be recognized as a systematic procedure 
to effect Gauss’ elimination method of solving simultaneous equations. 
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Since the logical operations of the triangulation method are straight- 
forward it can easily be programmed for digital computer solution. 
The number of multiplications and divisions required is approximately 
the same as that of Gauss’ elimination method. 

Example. Consider the solution of the set of equations in unknowns 
x, y, and z by the above method. 


x+2y+ z= 4 


w+ y- z= 5 OZ) 


¥— y+2z =-8 


Multiplying the first equation by —2 and adding to the second and 
multiplying the first equation by —1 and adding to the third will result in 


x+2y+ z= 4 
— By - 8z = -3 
-3y+ z2=-7 


Dividing the second equation by —3 we get 


x+2y+z= 4 
1 


y+2 


i 
1 
x 


- 3 +2 


Multiplying the second equation by 3 and adding to the third will result 
in 


x+2y+ z= 4 
Vo hi 
4z = -4 


The last equation gives z =—1. Having this, the second equation gives 
y = 2 and finally the first gives x = 1. 
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Solution of Simultaneous Linear Equations 
by Matrix Inversion 


Consider the set of n simultaneous equations in n unknowns: 


G41 X1 + GyQXq + GygXgzt+-++ + y,X, = dy 
y1X1 + Gg9Xp + GpgXgz + +++ + AanX_ = by 
: (1.73) 
GX + AjgXq + G;gX%y + +++ + Ajy,X, = b; 
44%, + AyoXq + AngXg + + Gn_X, = 5, 


where a’s are the constant coefficient, x’s are the unknowns, and 6’s 
are constants. The above equation can be written in matrix form as 
follows: 


[A] (x) = (B) (1.74) 


where [A] is a square n x n matrix, and (x) and (B) are column matrices. 
Matrices [A], (B), and (x) can be written 


Pals | tee “Caaeehemlh epee 2 « -|%2] G75) 


Premultiplying both sides of Eq. (1.74) by (A]"! results in 
(x) = [A]-1(B) (1.76) 


Thus the solution can be obtained by evaluating the inverse matrix 
[A}-! and multiplying it by column matrix B. 

In previous pages, numerical methods for the solution of simultaneous 
equations were described, where the number of multiplications required 
for solving a set of n equations in n unknowns was, approximately, n3/3. 
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In the following pages matrix inversion methods, to be used in conjunc- 
tion with the above method, will be described where n? operations are 
required to invert an n x n matrix. Thus, the method of Eq. (1.76) will 
involve three times more multiplications in inverting the matrix A than 
previous methods require in solving for a set of n unknowns. For this 
reason the method of Eq. (1.76) should be applied only when a set of 
x’s for a set of B’s are to be calculated. Under these conditions the 
inverse matrix A~! can be reapplied to calculate new x’s for new B’s. 


Augmented Matrix Method of Matrix 
Inversion 


The augmented matrix method of matrix inversion is based, primarily, 
on the Gauss elimination method of solving simultaneous equations. 
For this reason its derivation will be based upon a set of n equations 
and n unknowns. It will be shown that the procedure of inverting 
matrices by this method can be mechanized without relating it to a set 
of simultaneous equations. 

Consider the set of equations 


G41 X1 + AyQXq + +++ + Ay,X, = dy 
Gy1X%1 + ApoXy + --- + Gg, xX, = bi, 
2171 22°2 2n°n 2 
5 (1.77) 
@,1X1 + GygXq + +++ + GppX_ = by 


It is desired to find the inverse of the matrix of the coefficients [A]"}. 
This set of equations can be written in augmented matrix form as follows: 


A Ay xy 

G9, gg Gpg--- Gg, O 1 O ..- 0 : 
| 4 

om Gig «jg ++ Gin OO 6 O55 035540 = 

a by 

a, Gg ng ++ Gy, 8 0 0 atc: 1 : 

-b 


(1.78) 


28 Digital Computer Methods in Engineering 


where the large matrix is the augmented matrix of A. The idea of this 
method is to make all of the elements along the diagonal of the part A of 
the augmented matrix (a1, ayo, ..., @,,,) equal to unity and the rest of 
the elements of part A equal to zero. In doing this the A, part of the 
matrix will fill up, resulting in 


A A, “1 
SS Oe 

4 

1 zB 

1 : 

*n | _ 9 (1.79) 

Numbers -b; 

—by 

Zeros 5 

1 a 


Multiplying these matrices and transforming the (x) and (B) matrices to 
proper sides of the equal sign, we have 


* Ay 


—— by 
; by 
el Numbers ; (1.80) 
: b 
x n 


Comparing this to Eq. (1.76) we see that [A]-! = A,. Substituting the 
values of b’s from the given set of linear equations and multiplying the 
two matrices on the right of Eq. (1.80) results in values of the unknown 
x’s. Matrix A, in Eq. (1.79) is obtained from the augmented matrix of 
Eq. (1.78) by the following procedure. 

In order to follow the method described here, relations (1.78) should 
be considered as a set of equations rather than as two matrices multiplied 
together. Divide the first row, of the first equation, by a,,- Multiply 
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the resulting values of the first row by ay, and subtract from the second 
row; multiply the first row by a3, and subtract from the third row. 
Continuing this will result in a matrix of the form 


3 | 

- - = = O O-.. 0 Xo 

GO - = a ea 0 : 
Bae |r ees 

=, = = oe) -b; 

a —by 

ma = a, - 9 1 ~ 

-b 


where dashes and primes represent constants. Now, divide the second 
row by ayo. Multiplying the resulting row by aj.(i = 3, 4, ..., ») and 
subtracting from the third row and higher will give zeros in the second 
column. Continuing this operation on the third, fourth, and up to the 
nth rows will result in the matrix 


! \ x 
1 = 2 
1 4 
Bn |, 9. 82) 
—b; 
-by 
A ~T] 0, 


Working the above procedure backward from the nth row to the first 
row, i.e., multiplying the nth row by the element (n — 1, n) and 
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subtracting from the (n — 2) row, and so on, will produce zeros in the 
nth column. Repeating this process for the (n — 1)th row through the 
first row will result in the matrix 


: a 
1 ba 
*, 
Numbers = 0) 101¢83) 
-b, 
—by 
1 -b 
= 


Comparing this and Eq. (1.79), we note that the square matrix “numbers” 
is the inverse of the matrix of the coefficients (or = [A]~!). 

As mentioned before, the number of multiplications in inverting 
matrices by the augmented matrix method is estimated at n>? where n is 
the order of the matrix to be inverted. 

In order to crystallize the above procedures, let us consider the follow- 
ing numerical example. 

Example. Find the inverse of the matrix 


Bien di oh 
PR oud (1.84) 
i ee! gee 


by the augmented matrix method. In numerical evaluation the matrices 
(B) and (x) can be dropped and the mechanics of arithmetical operations 
described above can be carried out. The problem starts by writing 


ooo O.- tC 8 (1.85) 
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Dividing the first row by 2, we get 


il 3 1 0 1 0 (1.86) 


Multiplying the first row by —1, adding to the second row, multiplying 
the first row by —1, and adding to the third row will result in 


Zz 0.5 05 O05 0 0 
0 25 (0:5 °-0:5 1 0 (1.87) 
0 05 3.5 -05 0 1 


Dividing the second row by 2.5, multiplying the resulting row by —0.5, 
and adding to the third row, we get 


1 0.5 #05 05 O 0 
1 0.2 -0.2 04 O (1.88) 


oo 


0 3.4 -04 -0.2 1 


Dividing the last row by 3.4, multiplying the resulting row by —0.2, 
adding to the second row, multiplying the third row by —0.5, adding to 
the first row, we get 


1 0.5 0 0.5588 0.0294 -0.1470 
0 1 0 -0.1764 0.4118 -0.0588 (1.89) 
0 0 1 -0.1176 —0.0588 0.2941 


Multiplying the second row by —0.5 and adding to the first results 
in 
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1 0 O 0.6471 -0.1765 -—0.1176 
0 1 O -0.1764 0.4118 —0.0588 (1.90) 
0 O 1 0.1176 -0.0588 0.2941 


From this the inverse matrix of the original matrix will be 


Den eect 0.6471 -0.1765 -0.1176 
ne: ala =| -0.1764 0.4118 -0.0588 (1.91) 
ion le -0.1176 -0.0588 0.2941 


FORTRAN Program for Augmented Matrix 
Inversion Method 


The following FORTRAN program is written for matrix inversion by 
the augmented matrix method discussed above. The data used in the 
program are the same as the example problem above. The computer 
program output is given at the end of the program listing. 

In order to understand the statements of the program, the reader 
is referred to [1]. The following paragraphs give a general explanation 
of the program. 

Statement 10 reserves computer memory location for a matrix of 
100 by 100. Statements 20 and 30 read the order of the matrix to be 
inverted, N = 3, and the matrix coefficients S(I, J), starting with the first 
row and proceeding to the second and third rows, as given in Eq. (1.84). 
Statement 40 and the following two statements set values to be used 
later. Statements 50 through 80 form the augmented matrix as given by 
Eq. (1.85). Statements 90 through 220 transform the augmented matrix 
into the form given by Eq. (1.88) with the last row divided by 3.4. 
Statements 230 through 290 transform the matrix into the form given by 
Eq. (1.90). Statements 300 through 330 print out the inverse matrix 
as given at the end of the program and also by Eq. (1.91). 


c MATRIX INVERSION BY AUGMENTED MATRIX METHOD 
10 DIMENSION S(100,100) 
20 DATAN/3/ 


30 
40 


50 

60 

70 

80 

90 
100 
110 
120 
130 
140 
150 
160 
170 
180 
190 
200 
210 
220 
230 
235 
240 
250 
260 
270 
280 
290 
300 
310 
220 
330 
340 
350 
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DATA ((S(I, J), J=1,3),1=1,3)/ 2.,1.,1.,1.,3.,1.,1.,1.,4./ 
-1 


NX=N+1 

NY=2«N 

DO 80 J=NX,NY 
S(1,J)=1.0 

l=1+1 

CONTINUE 

L=1 

K=2 

XM=S(L,L) 

DO 140 J=L,NY 
S(L,J)=S(L,J)/XM 
CONTINUE 

DO 190 I=K,N 
X=S(I,L) 

DO 190 J=L,NY 
S(1,J)=S(1,J)—S(L,J) *X 
CONTINUE 

L=1+1 

K=K+1 

IF(L—N) 110,110,230 
L=N 

LZ=L-1 

DO 290 K=1,LZ 
l=L—K 

Y=S(I,L) 

DO 290 J=L,NY 
S(1,J)=S(1,J)—S(LJ)*¥ 
CONTINUE 

L=L-1 


IF(L—1)320,320,235 

WRITE(2,330) ((S(1,J),J=NX,NY),1=1,N) 

FORMAT (15H INVERSE MATRIX / (3X1P3E10.3)) 
STOP 

END 


Computer solution obtained: 


INVERSE MATRIX 
6.471E—01—1.765E—01—1.176E—01 
-1.765E-01 4.118E—01—5.882E—02 
—1.176E—01—5.882E—02 2.941E—01 
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Inversion of a Matrix by Partitioning 


The inversion of large matrices can be accomplished by partitioning 
the matrix into submatrices. This method reduces the order of the 
matrices to be inverted, thereby reducing the computer memory storage 
requirements. Consider the nth order matrix 


AIB 
M = C+") (1.92) 
Gep 


partitioned into four submatrices A, B,C, and D where A and D are square 
matrices of order p and q, respectively; p + q = n. The inverse matrix is 
denoted by 


: W : x 
M-* = —+:) (1.93) 


where W and Z are square matrices of orders p and g. Using the 
property of inverse matrices 


Al B\/Ww |X iD 
EC) 
c!lpo/\y!z o 1 


where / is the unit matrix and multiplying Eq. (1.94) we get 


AW + BY = 1 
AX + BZ = O 
(1.95) 
CW+DY =O 
CX + DZ =1 


Multiplying the third equation by BD~! and subtracting from the first, 
we obtain 


(A=-BD* OW = I (1.96) 
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or 
W = (A- BD-!¢)* (1.97) 
From the third equation we also find that 
Y = -D-'cw (1.98) 


From the second and fourth equations we obtain 


(1.99) 


From the above equations, it is seen that the inversion of a matrix of 
order n reduces to the inversion of two matrices of order p and two of 
order q and several matrix multiplications. 

The equations given above can be altered so that for computation of 
matrices W, X, Y, Z, only two matrices need be inverted. In terms of the 
inverse of the square submatrix D, we get 


= (A= BD Saher 


= -WBD-! 
= -D-1cW 
= D-! = DCX 


(1.100) 


NS pe OS 
i 


The above equations can also be set in terms of A7 matrix, as follows: 


= (DsCACLB): - 
= —ZOA1 
-A'BZ 
=vAs) = AP BY 


(1.101) 


Ss ON 
il] 


Note that Eqs. (1.100) and (1.101) involve the inverse of matrices D and 
A since they were the only square matrices assumed in the partitioning 
process. In the partitioning method of matrix inversion, since a great 
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number of calculations are involved, it is expedient to check the inverse 
matrix by multiplying it by the original matrix and obtaining the unit 


matrix. 
Example. Find the inverse of the third-order matrix 


5 (1.102) 


by partitioning, as indicated. 
Partitioned matrices A, B, C, and D of Eq. (1.92) can be written 


ore aia 9.6 (1.103) 
A= B= ( ) C = (1.0,3.2) D = 6.7) 
OHS 4.5 


From this, the inverse of matrix D of Eq. (1.100) will be 


pt = 1 ~ 0.14995 (1.104) 
6.7 


The matrix W of Eq. (1.100) will become 


2 25 3.8 1.43280 4.58496] {1 
~\t-21 13] [0.67162 2.14918 
( 1.06716 Rae 

-2.77164 -0.849254 


(1.105) 


( 0.275531 - ae 
-0,89923  -0.34623 


Having the above matrix, the second of Eqs. (1.100) can be used to 
calculate X. This results in 
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( 0.275531 ree |e 
—0.89923 —0.34623 4.5 


c ot 

~ \ 1.52099 

Using the previously calculated values, the third and fourth relations of 
Eq. (1.100) will result in values of Y and Z as follows 


) (0.14925) 


(0.388359 0.20338) 
—0.543799 


Setting the values W, X, Y, and Z in Eq. (1.93) of the inverse matrix, we 
get 


0.275531  -0.254709 -0.223718 
Ml = |-0.89923  -0.34623 1.52099 
0.388359 0.20338 «= - 0.543799, 


(1.106) 


which is the required answer to the problem. 


Gauss-Seidel Method of Solving 
Simultaneous Equations 


A system of linear equations is called diagonal when the coefficient of 
the diagonal element dominates, that is, the absolute value of this coef- 
ficient is larger than the sum of the absolute values of the rest of the 
coefficients on that row. The simultaneous equations arising in many 
physical systems satisfy this condition. In these cases the iterative method 
of Gauss-Seidel can be applied for the solution of the unknowns. 

The mathematical derivation of the convergence criteria of the Gauss- 
Seidel iteration method for solving sets of simultaneous equations is 
complicated as the criterion of convergence is related to eigenvalues (see 
Chap. 2) of a matrix derived from the matrix of the coefficients, matrix 
A in the following discussion [7]. It will be seen in the succeeding 
chapter that the computation of eigenvalues for high-order matrices is a 
difficult numerical problem. For this reason, in practical cases, the 
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convergence criterion of Gauss-Seidel method can be applied only in the 
case of sets of simultaneous equations consisting of very few equations— 
PATO) ge 

Given the set of simultaneous equations 


211%, + GjQX% + +++ + y_% = 0) 
@y1X1 + AgopXy + +++ + Ay x = DB 
a1 J pple Ne} 2n*n i (1.107) 
@,1%1 + GngXo + +++ + GypX, = b, 
or, in matrix notation, 
Ax = B (1.108) 
Equations (1.107) are solved for the diagonal unknowns 
2 b (2 “13 | “Any 
YS ae ae oY pe ets 
3 34 1 Pat 
ee: (2 ee is ie 
Bee i ee a aE oe ae 
To \ a2 425 GQ9 (1.109) 
. b, (ts : a, go Payal, ) 
(itm rests Wma 35 n-1 
aan ann Inn Gin 
and in matrix notation 
Cre (Bose (1.110) 


where C is a matrix with diagonal elements of zero. Gauss-Seidel 
iterations start by assuming that x, = X3 = +++ = x, = 0 and solving the 
first of Eqs. (1.109) for x, using this value of x, with Xj =X%yaeee = 
x, = 0, solving the second of Eqs. (1.109) for X , and so on. After 
obtaining the set of values for x, through x,,, the second iteration starts 
by using these values in Eqs. (1.109). In most practical cases solution is 
terminated where the values of the unknowns from one iteration to the 
next do not change within a specified number of significant figures. 
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Example. Solve the set of equations 


25x, + 2x9 + xg = 69 


2x, + Wry + xg = 63 Orr) 


X, + X%_ + 4x3 = 43 


The solution starts by putting the above equation in the form (1.109) 
and carrying out the above outlined procedure. The values of x at the 
end of the first iteration are 


%, = 2.76 X_ = 5.748 xg = 8.623 


The solution will converge after five iterations to the values 


Computer Program for the 
Above Solution 


The following digital computer program in BASIC programming 
language was written for the Gauss-Seidel iteration method. Although 
most of the computer instructions are self-explanatory, the reader is 
referred to the BASIC programming language manual [2] for further 
explanation of instructions. 


100 DIM A(10,10),C(10,10),B(10),X(10) 
110 READN 

120 PRINT “COEFFICIENT” 
130 FOR I=1TON 

140 FOR J=1TON 

150 READ Ai(I,J) 

160 PRINT A(I,J); 

170 NEXT J 

180 PRINT 

190 NEXT I 

200 FOR I=1TON 

210 FOR J=1TON 

220 LET C(I,J)=A(I,J)/A(1,1) 
230 NEXT J 
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240 LET C(I,I)=0 

250 NEXT | 

260 PRINT “RIGHT SIDE” 
270 FORI=1TON 

280 READ Bi!) 

290 PRINT B(I); 

300 NEXTI 

310 PRINT 

320 FOR I=1TON 

330 LET X(I)=0 

340 NEXT! 

350 LET L=1 

360 PRINT “ITERATION VALUES OF X(1) THRU X(N)” 
370 FOR l=1TON 

380 LET D=0 

390 FOR J=1TON 

400 LET D=D+C (I,J)*X(J) 
410 NEXT J 

420 LET X(I)=(B(I)/A(I,1))—D 
430 NEXT! 

440 PRINT L; 

450 FOR l=1TON 

460 PRINT X(I); 

470 NEXTI 

480 PRINT 

490 IF L=10 THEN 570 
500 LET L=L+1 

510 GOTO 370 

520 DATA3 

530 DATA 25,2,1 

540 DATA2,10,1 

550 DATA 1,1,4 

560 DATA 69,63,43 

570 END 


Statement 100 of the program reserves computer memory spaces for 
the coefficients of Eq. (1.108). The matrix C of Eq. (1.110) is also 
shown in the dimension statement 100. Statement 110 reads the number 
of the simultaneous equations. Statements 130 through 190 read the 
coefficients of the unknowns of Eqs. (1.107). Statements 200 through 
250 compute the terms of the matrix C of Eq. (1.110). Note that the 
diagonal elements of this matrix are set to zero by statement 240. 
Statements 270 through 300 read the elements of the right side of Eq. 
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(1.107). Statements 370 to 430 compute the values of x’s at each 
iteration according to Eq. (1.110). Statements 450 through 470 print 
out the answers. Statement 490 limits the number of iterations to 10. 
The output of the program for 5 iterations of the above example problem 
is as follows: 


COEFFICIENT 

25 2 1 

2 10 1 

1 1 a 

RIGHT SIDE 

69 63 43 

ITERATION VALUES OF X[1] THRU X[N] 
1 2.76 5.748 8.623 
2 1.95524 5.04665 8.99953 
3 1.99629 5.00079 9.00073 
4 1.99991 4.99995 9.00004 
5 rs 5. 9. 


In this chapter we have discussed the introductory material on matrices 
and linear simultaneous equations. However, we have not treated the 
question of round-off errors in the computation and the stability of 
solutions. There are a number of texts which discuss these problems 
from a mathematical point of view [4, 7]. In most engineering appli- 
cations, however, these questions are studied by an examination of the 
answers obtained for a variety of input parameter conditions. For 
example, the simplest method of examining the correctness of the inverse 
matrix A is to multiply it by A and obtain the unit matrix. The 
question of the stability of solution, for example, for matrix inversion, 
can be answered by perturbing one or some of the matrix elements 
slightly and examining its effect on the solution. 


PROBLEMS 
1. Given the two equations 


ax + 3y = 0 
2x +(a-l)y = 0 


for what value of a do these equations have a nonzero solution? 
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Find the value of the ratio x/y for each value of a. 
2. Expand the determinant 


3. Evaluate the determinant 


TOC 2 
D={|20 4 6 
30 12 9 


4. Verify that x = 2 isa root of the equation, 


(10 — x) -6 2 
-6 (9 — x) -4 =0 
2 -4 (5 — x) 


5. Evaluate the determinant 


16> “IPSS S85 
ae i So | 
-10 -3 2 1 
Die Get de ni 
by the method of Chid. Answer: —1820 
6. Evaluate the determinant 
T G4 
3-4 #5 
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by following the logic of the FORTRAN program given for determi- 


nant evaluation. Answer: 25. 
Show that 
2 cos 0 1 0 
D= 1 2 cos @ 1 = sin 40 
sin 0 
0 1 2 cos 0 


provided that @ is not a multiple of 7. 
Solve the equations 


9x - 17y + 21z = 66 
13x + 8y - Tz = 32 
ix - 8y - 9z = -37 


Answer: x = 2.818, y = 3.821, z= 5.029. 
Find the values of a for which the equations 


(1 - a)x - Gy + 4z = 0 
—6x +(2-a)y — 2z = 0 
4x - 2y - (3 + a)z = 0 


can be satisfied by values of x, y, z, not all zero. For each value of 
a, find the ratios x:y:z. 

Solve the set of equations for x), x2, and xz by the method of 
elimination of Gauss 


H 
© 


7x1 + 2x9 + 4x3 = 
8x, -4x9 + 5xg = -14 

x1 + 3x9 - 2x, = 12 

Answer: x, = 1, %9 = 3, xg = -1. 


Solve Prob. 10 by triangulation method. 
Show that the equations 
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13. 


14. 


1S; 


16. 
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N+ y+ez =m 
2x + 8y + 4z2 = 0 
3x + 4y + 5z = 1 


have a unique solution if c # 1 and find it. Discuss the case ¢ = 1. 


Prove that 
a Do Ae I me 3 ip a eee 
Cc) Sih 2 Zz co yilles ly P @ 
i 70 Sa we £ go Pp 
where p = ax + by + cz, q = az + bx + cy,r = ay + bz + cx. 
Verify that 
=f & 8 
2-3 3 5 
-0 
2-3 2 9 
1 2-2 2 
If 
1 =2 
2. 3 7 
A = B=| 3 0 
0. 2-1 
-1 2 
find AB and BA. 
If 


show that AB =0. This shows that from AB = 0 we cannot deduce 
that A=0 or B=0. 


17. 


18. 


19. 


20. 


21. 
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show that A2 = 0, and hence that we cannot deduce from A? =0 
that A =0. 

Show that the commutative law holds for the product of two 
matrices of the form 


a b ei aey 
A = and B = 
-b a -y x 


If A and B are square matrices of the same order, show that, unless 
AB = BA, 


a. (A + B)(A - B) # A? - B2 
b. (A + B)? # A? + 2AB + B? 
c. (A - 2B)(A - 3B) # A® - 5AB + 6B? 


If 


If 


Peete a pik 
Ae Lvl 2 
0-1 6 


show that A is singular. Construct adj A. 
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22. 


23. 


24. 


25. 


26. 


2s 


28. 
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If 


nO Fi de 3 Be =2i 6 10 
A=|2 5 5] showthat Aq! = é ee edt ages! 
oy 1b. LL Cesk. W 


If: 
bet. 2 sia: | 
A=|1 a a?| then Aq! = ; re 
1 @ a@ ta @ 


where a = exp (27i/3), i = (-1)'”?, 
If A is a square matrix of order n and if x is a scalar, show that 


|Ax| = |Alx” and that adj (Ax) = (adj A)x"~ + 


Obtain the inverse of the matrix 


2) 8 
[APS | aes 
"saa gad 


by the augmented matrix method. Check your solution by multi- 
plying the inverse matrix [A]~! by matrix [A]. 

Follow the logic of the FORTRAN program given for the aug- 
mented matrix method in inverting the matrix of Prob. 25. 

Verify Eqs. (1.100) and (1.101), which were used to obtain inverse 
matrices by matrix partitioning. 

Obtain the inverse of the M matrix 


29. 


30. 


31. 


32. 


33) 
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25 38 9.6 
M=]-21 13 45 
LO 8:2) 67 


by partitioning the matrix and using Eqs. (1.101). 

Obtain the inverse of matrix of Prob. 25 by the matrix partition- 
ing method. 

Solve the set of simultaneous equations 


69 


25x, + 2X9 + X3 
2x, + 10xg + xg = 63 
Xp + X_ + 4xg = 43 


by the matrix inversion method. 
Solve the set of equations 


2x, - 8xg + xg = -4 
X, + Xg- 2x3 = -7 


-X; + Xg+ X%3 = 9 


by the matrix inversion method. 
Solve the set of simultaneous equations 


4x, + X_9- Xg = 17 
X1 + 6xg - 2x3 = 14 
X1+ X%2- 5x3 = 1 


by the Gauss-Seidel method. 
Solve the set of equations 


8x1 + X_2+ Xj = 4 
X, + 4x9 - x3 = -5 
X1 + X%_ - 6xg = -12 


by the Gauss-Seidel method. 
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2 EIGENVALUES, 
EIGENVECTORS, AND 
QUADRATIC FORMS 


The study of many physical systems of practical importance, such as 
the investigation of the elastic vibrations of a bridge or any other solid 
structure, the flutter of an airplane wing, the transient oscillations of an 
electric network, or the buckling of an elastic structure, leads to the 
solution of what is known in the mathematical literature as an eigenvalue 
problem. A general mathematical discussion of the eigenvalue problem 
and its ramifications is given in this chapter. This class of problems is 
also referred to as characteristic value problems. The terms “eigen” and 
“characteristic” are used interchangeably. 


THE EIGENVECTOR (CHARACTERISTIC 
VECTOR) EQUATION 


Let us assume that we are given an nth order square matrix A and 
an nth-order column vector X. Then A and X are conformable matrices 
49 
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and can be multiplied in the order AX to yield another vector Y, that is, 
AX = Y (2.1) 


In general, the vector Y will not have the same direction or magnitude as 
the vector X. Let us ask the question of whether or not it may happen 
that the vector Y has the same direction as the vector X. If this is the 
case,Y = AX where A is a multiplicative scalar. If we place Y = AX in Eq. 
(2.1), we obtain 


AX = AX (2.2) 


Equation (2.2), usually called the matrix eigenvector equation, arises in 
many branches of applied mathematics. The main problem is to find the 
values of A and the corresponding vector X for which Eq. (2.2) holds. 
Before considering the general case, let us consider the case of a second- 
order square matrix A given by 


a b xy 
A= and X = (2.3) 
Coad Xo 
If we expand the eigenvector equation (2.2) in this case we obtain 
a 0b] [x, xy 
=A (2.4) 
c d| |x» Xo 


If we multiply the matrix A by the column vector X as indicated in 
Eq. (2.4), we obtain the two scalar equations 


ax, + bxg = Axy 
= (2.5) 
cx) + dxy = My 
Equations (2.5) can be rewritten in the form 
(a — A)x, + bxy = 0 
1 2 (2.6) 


cx; +(d- Ax» = 0 
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Equations (2.6) are a set of homogeneous equations of the type discussed 
in Chap. 1. In order for these equations to have nontrivial solutions for 
x, and x9, it is necessary that the determinant of the coefficients p (A) 
vanish, that is, 


a-A b 
pir) 


ic d-.X 


(2.7) 
det |A — Al| 


" 


=0 


The matrix K(A) = [A — Al] is called the characteristic matrix of the 
matrix A; the polynomial p(A) = det (A — Al) is called the characteristic 
polynomial. Equation (2.7) or p(A) = 0 is the characteristic equation 
of the matrix A. 

THE EIGENVALUES OF A 


The characteristic equation (2.7) of the matrix A has the form 


pir) = r2 - A(a + d) + (ad — be) = 0 (2.8) 


The two roots A, and A, of this quadratic equation are the eigenvalues 
of the second-order matrix A given by Eq. (2.3). 

If A, and Ag are the roots of Eq. (2.8), we can write the polynomial 
p(d) in the following factored form: 


p(A) = (A= A,)(A = Ag) = A — AA, + Ag) + AYAQ = 0 (2.9) 
If we now compare Eqs. (2.8) and (2.9), we see that the coefficient 


of the —A term (a + d) of Eq. (2.8) must be equal to the coefficient of the 
~A term of Eq. (2.9), and we must therefore have 


Ay + Ag = (a +d) = TrA = Trace of A (2.10) 


The constant term of Eq. (2.8) must correspond to the constant term 
of Eq. (2.9); therefore, we must have 


Ay Ag = (ad - be) = det A = determinant of A (2.11) 
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The important fact that the sum of the eigenvalues of a matrix A 
equals the sum of the principal diagonal elements of A or the trace of A 
and that the product of the eigenvalues of A equals the determinant of 
A is true not only for second-order matrices, but is true also in the 
general case of a matrix of order n. This fact will be established later. 
The two eigenvalues can be obtained by solving the quadratic equation 
(2.8) (the characteristic equation of A). 


THE EIGENVECTORS OF A 


The eigenvalues of A are the numbers A, and Ay. In order to obtain 
the eigenvectors of A that correspond to these eigenvalues, we return 
to the simultaneous equations (2.6). From these equations, we may 
obtain the ratio x5/x, in the form 


x, 
Bae (2.12) 
x} b A-d 
If we place A = A, in Eq. (2.12) we obtain the ratio 
Ay -a@ 
= (2.13) 
b 


We thus obtain x) = r,x,. This fixes the ratio of x, to x, and therefore 
the direction of a certain vector. If we place A = Ag in Eq. (2.12) we 
obtain 


Ag - a 
b 


(2.14) 


To = 


We thus obtain x, = rjx,, and therefore another direction of a vector. 
We can write X, for a vector of undetermined length having the direction 
r,, and X, for a vector of undetermined length having the direction To 
and obtain 


(2.15) 
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The vector X, is then said to belong to the eigenvalue A, and the vector 
X, to the eigenvalue Aj. Since the quantity x, is not specified, only 
the directions of these vectors are fixed. If we choose x, to be given by 


A Pye! and a (2.16) 
Q+4)” (1 +3)” : 


in X, and Xp, respectively, the vectors will then be normalized to have 
unit length. 

It is seen from the above discussion that the eigenvector equation 
gives eigenvalues and eigendirections. In most discussions involving eigen- 
values the vectors are normalized to unit length and in such case they are 
uniquely defined. 


THE EIGENVALUES AND EIGENVECTORS 
OF AN nTH-ORDER SQUARE MATRIX 


If we expand the eigenvector equation (2.2) in the general case in 
which the matrix A is a square matrix of the nth order, we have 


G41 X1 + GyQXq + +++ + Gy _Xp_ = Ay 
1X1 + AgQXp + +++ + Gp_X_ = AXQ (2.17) 
G4%1 + AngXg + +++ + GypXp, = AX, 
This set of equations can be written in the form 
(ay, — NX, + GyoX%_ +++ + a4,X%, = 0 
“211 + (doo - AN) X_ tose st GonX_ = (2.18) 
04%, + G,9% + 2+ + (75 NX, = 0 
In matrix notation, Eq. (2.18) can be written in the compact form 
[A — A(X) = 0 (2.19) 


where | is the unit matrix of order n. The homogeneous equations (2.18) 
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have a nonzero solution only in the case where the determinant p(A) of 
the coefficients vanishes. We therefore have 


p(A) = det [A - NI] = 0 (2.20) 
In matrix algebra, the matrix 
K(A) = [A - AJ (2.21) 


is called the characteristic matrix of the matrix A. The determinant of 
the characteristic matrix p(A) is the characteristic polynomial of A. This 
determinant is generally of order n and therefore is a polynomial of the 
nth degree in A. The equation p(A) = 0 is the characteristic equation of 
the matrix A. 


FUNDAMENTAL PROPERTIES OF THE 
CHARACTERISTIC POLYNOMIAL OF A 


If A is an nth order matrix the characteristic polynomial can also be 
written in the form p(A) = det{Al - A] = 0. In this form it results in 
the characteristic equation with A” coefficient of unity, 

pid) = AM 4 aA" 4 aan" 2 4... 4a, ;A4a, = 0 (2.22) 
Setting 4 = 0 in Eq. (2.22) will result in 
p(0) = a, (2.23) 
However, since p(A) = det [AJ — A], we have 
p(0) = det [-A] = (-1)"det[A] (2.24) 
If we substitute Eq. (2.23) mto Eq. (2.24), we obtain 
a, = (-1)"detA (2.25) 
If we assume that the matrix A has n distinct eigenvalues 


(Ay, Ag, ---,A,), then since the eigenvalues are the roots of the charac- 
teristic equation, we can write Eq. (2.22) in the factored form 
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P(A) = (A — Ay) (A = Ag)... (A - AL) (2.26) 


If we place A = 0 in Eq. (2.26), the result is 


p(O) = (-D"Ay Ag Ag .-- Ap (2.27) 
If we compare Eqs. (2.24) and (2.27), we have 
det [A] = A, Ag... A, (2.28) 


Therefore the determinant of the nth order square matrix A is equal 
to the product of the eigenvalues of A. It follows that if det [A]= 0 so 
that A is a singular matrix, then at least one of its eigenvalues is zero. 

If the factored form of p(A) of Eq. (2.26) is multiplied out, it will be 
found that the coefficient of 4”~! which is a, in Eq. (2.22) is given by 


Ay + Ag t Ag ters +A, = -Qy (2.29) 


If, on the other hand, the determinant, det [A - Al], is expanded to 
determine the coefficient of a, or a”-1 in the polynomial p(A), the 
result is 
@, = ~(4); + Ag + G33 + +++ + 4,,,) = —Trace A (2.30) 
since by definition the trace of the matrix A, TrA, is the sum of the 
principal diagonal terms of A. If Eqs. (2.29) and (2.30) are compared, it 
is seen that we have the important relation 
Trace A = Ay + Ag+ Ag ters +A, (2.31) 
The above equations can be used primarily as a check on computation 


of the eigenvalues. The nth-order polynomial of Eq. (2.22) can be solved 
for characteristic values by the methods outlined in Chap. 3. 


NEWTON’S FORMULAS FOR THE COEFFICIENTS 
OF THE CHARACTERISTIC POLYNOMIAL 


We have seen that the coefficient a, of the characteristic polynomial 
p(d) is given by the relation a, = detA and that the coefficient 
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a, =—Trace A. The mathematician M. Bécher has given a set of equations 
for the coefficients of the characteristic polynomial p(A).* 
We define the numbers S,, Sy, S3, ..., S,, by the equations 


S; = Tr(A) S, = Tr(A%) S, = Tr(A%)... S, = Tr(A”) (2.32) 


so that S, is the trace of the kth power of the given nth-order square 
matrix A. Bécher has shown by a long algebraic argument based on 
Newton’s formulas that the coefficients a), aj, ..., a, of the character- 
istic polynomial p(A) of A can be computed successively by the following 
recurrence formulas: 


a, = -S; 
a,S, + Sy 
eer 
> 2 
— ESD + aS) + S3 (2.33) 
? 3 


As a numerical example of the use of these formulas, let it be required to 
determine the characteristic polynomial and therefore the characteristic 
equation of the matrix 


2-1 0 
Boa) Oo) 4°35 (2.34) 
-8 O -3 


We first compute A? and A? and obtain 


*M. Bocher, “Introduction to Higher Algebra,” pp. 243-244, Dover Publications, 
Inc., New York, 1964. 
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=5 -6 =6 eiog 191" =18 
y coe (ayer eam) ol] oy A | (2.35) 
Saas. Oo 16624 «(t 


We then compute the traces 


8, =TrA=8 5S, = Tr(A?)= 11 Sy = Tr(A*) = 27 


(2.36) 
By the use of Eq. (2.33) we then obtain 
a = -S; = -3 
a,S, + Sg 
@ = -———— = =e | 
aD (2.37) 
a2S, + aS, + Ss 
a, = - ———___—_—_- = 3 
3 
The characteristic equation of the matrix (2.34) is, therefore, 
2 827 A 43) = 0 (2.38) 


Equation (2.25) in this case gives detA =—3. 

It should be noted that although Boécher’s formulas represent an 
elegant mathematical relation, in practical cases they are not used in 
computation of the coefficients of the characteristic polynomial of high 
order (3d and higher) matrices. The reason for this is that raising 
matrices to various powers, in addition to requiring a great number of 
calculations, in many cases results in loss of accuracy as the elements of 
the new matrices require more significant figures. The following numeri- 
cal method, due to A. N. Krylov, can be used for the determination of 
the coefficients of the characteristic equation. 


THE METHOD OF A. N. KRYLOV FOR 
DETERMINATION OF THE CHARAC- 
TERISTIC EQUATION 


Consider the matrix 
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A =| Sar “Seam “Cape an (2.39) 


whose characteristic equation is desired. As discussed before, the char- 
acteristic equation results from setting the determinant 


My -A yp ++ Gy 
pa =| 721 Cia eras (2.40) 
a4 a2 Jog Gaim 


equal to zero. The method of A. N. Krylov consists of reducing the 
determinant of Eq. (2.40) to the form 


bay - bap s+ Ban} _ 9 (2.41) 


where the characteristic value A appears only in the first column. 
Expanding the above determinant in terms of its first column will give 
the characteristic equation in powers of A. Note that the form of Eq. 
(2.41) is such that the characteristic polynomial can more readily be 
obtained than from the expansion of the determinant of Eq. (2.40). 

To derive the above relation we start by noting that Eq. (2.40) is the 
necessary and sufficient condition for the system of homogeneous 
equations 


AX, = G44 %X + GyQX_ +--+ + 44,X, 


Mig = Gy1X1 + AyoXy + +++ + Ag_Xpy 
(2.42) 


x = G,1X1 + AngXo + -++ + G,,X%, 
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to have a solution x,, x9, x3, ..-, x, different from zero. Multiplying the 
first equation by A and replacing the right side values of Ax;, Axg, Axg, --- 
of the resulting equation by their equivalents from Eq. (2.42), we get 


Mx, = bo) X1 + bogX%y_ + +++ + bon*n (2.43) 


where 


Multiplying Eq. (2.43) again by \ and replacing the right side of the 
resulting equation by equivalent values from Eq. (2.42), we get 


xy = bg,X, + bgpX%_ + +++ + b3n Xn (2.44) 


Repeating the above process will result in the set of equations 


Ax, = by, xy + Dygx%Q + +--+ DYAx, 
Dds. wit bea Racine Pork kis, beet 
1 2171 2272 2 

3 ee (2.45) 
NX, = by Xy + DygXy + +++ + OnnXn 

where 
bin = Ap 
. a Bey ea agitt 

eerie cs 

k =1, 

i S i ssk bieats <n 


The set of equations (2.45) will have a nonzero solution in x’s only if 
the condition on the determinant of Eq. (2.41) is satisfied. 
Expanding the determinant of Eq. (2.41) about the first column and 


denoting the cofactors of element (i, j) by C; j> We get 
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Cc, 44" + Cea + Cae tore 


5 (2.46) 
+ Cy yr- » b,Cy = 0 
i=l 
Dividing through by the coefficient of the 4” term, we get 
M+ py yA" 1 +p, oA" 2 + +++ + DIA + Dg = 0 (2.47) 
where 
Cc, 
P; = mds j=1, snul 
Cay 
n 
Cc; 
il 
i ay bi 
i=1 Cha 


In Eq. (2.47) the p’s are the coefficients of the characteristic equation of 
the matrix A of Eq. (2.39). Having obtained this equation the roots can 
be evaluated by root finding methods of Chap. 3. It should be noted that 
in many physical problems a great deal of information can be extracted 
from the characteristic equation without actually solving for the roots. 

Example. It is required to find the characteristic equation of the 
determinant 


2 4 -6 
Av= | 4 2 -6 
-6 -6 -15 


From Eqs. (2.45) the values of the coefficients b,;; can be calculated 
by, = @,-=2 by = Go = 4 big = G3 = pect 


and 
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3 
bai = y by 5451 = 511411 + P1241 + b13431 = 56 


Similarly 


bajo iBagam tl 0S y =i, 458 


Having these values, the determinant corresponding to Eq. (2.41) can be 
written 


2a) ee -6 
D,() =| 56-2 52 54 
-4-23 4 = -1,458 


The cofactors of elements (1, 1), (2, 1), and (3, 1) can be calculated as 
follows 


Cy, = (52)(-1,458) - (54)(4) = —76,032 
Cy, = —((4)(-1,458) - (-6)(4)) = 5,808 
(4) (54) — (-6)(52) = 528 


cS 
oo 


The summation term of Eq. (2.46) can be calculated from the above 
values 


>) by Cj, = — (by Cy + B91 Cay + 431 C31) 
i=l 


171,072 
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Using the calculated values in Eq. (2.46), we get 
528A3 + 5,808A2 — 76,032A - 171,072 = 0 
After dividing by the coefficient of A3 term, we have 
a3 4 11d? - 144d - 324 = 0 
As mentioned before, the roots of this equation can be obtained by 
root-finding methods of Chap. 3. The characteristic vectors corresponding 


to these roots are obtained by solving the set of simultaneous equations 
(2.42), 


DETERMINATION OF THE EIGENVECTORS OF A 


As we have seen, the eigenvector equation AX = AX can be written 
in the form 


[A - A(X) = K(A)(X) = 0 (2.48) 
where K (A) is the characteristic matrix of A. 

To each eigenvalue A = A;, there corresponds a column vector X; 
called an eigenvector. The eigenvector X;, is said to belong to the eigen- 
value \,; it satisfies the equation 

BGA Kee 0 Gia ly O chases a (2.49) 
Equation (2.49) gives nonzero solutions for the numbers in the column 
X, since detK(A,) = 0. 

As a consequence of the definition of inverse matrix, we have 

K(A) adj K(A) = det K()I = pl (2.50) 

If we place A = A; in Eq. (2.50), we have 


K(A) adj K(A) = pAayl = 0 (2.51) 


This result follows from the fact that p(A,) = 0, since A, is an eigen- 
value of A. If we now compare Eqs. (2.49) and (2.51), we conclude that 
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X, can be taken to be proportional to any row of adjK (A,); we can then 
write 


X, = (const)(any column of adj K (A;)) (2.52) 


By the use of Eq. (2.52) we again obtain an eigendirection, as was the 
case in the simple situation of the previously given second-order square 
matrix. 


c 


a- xz b 
K(A) = (2.53) 
e d- xX 


We now take the transpose of K (A) and obtain 


KO = ' Seats (2.54) 


te 
In that simple case, we had A = if in this case we have 


The adjoint of K (A) is therefore 


d-x -b 
adj K(A) = (2.55) 


-c a-xr 


Then by the use of Eq. (2.52) we have 


d- 2X -b 
X, = (const) or X, = (const) (2.56) 
=: a-arA 


Similarly, the eigenvector X, is proportional to either one of the columns 
of adjK (Az). This is the same result that we obtained previously. 

The result (2.52) is true in general and enables one to obtain the 
eigenvectors of a matrix whose eigenvalues are distinct. It sometimes 
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happens that the characteristic equation (2.22) has repeated roots. In 
this case the matrix A is said to have multiple eigenvalues. We shall 
reserve the discussion of multiple eigenvalues for a later section. For 
the moment, we will consider the case for which the characteristic 
equation (2.22) has distinct roots and therefore the matrix A has n 
distinct eigenvalues Aj, Ag, Ag, -.., A,- In such case the eigenvectors or 
eigendirections are given by Eq. (2.52). 


PROPERTIES OF THE EIGENVECTORS OF A 
SQUARE MATRIX 


In this section we shall discuss several of the most important properties 
of the eigenvectors and eigenvalues of a square matrix A. These properties 
are of great theoretical and practical importance. 

Property 1. Let A be a symmetrical matrix, so that a;; = a;; or 
A’= A. If the eigenvectors X, and Xx; belong to the eigenvalues A, and 
Aj. where A; # Ajo then X; and X; are orthogonal vectors. That is, their 
scalar product vanishes, or 


Hex, = GX, = 0 (2.57) 


F oa 3 


By definition, X; satisfies the eigenvector equation 
AX, = A;X; (2.58) 


If we take the transpose of Eq. (2.58) and use the reversal law of 
transposed products, we obtain 


XA = A,X; (2.59) 


t 5 ines 
If we postmultiply Eq. (2.59) by X;, we obtain 
X;AX; = A;X;X; (2.60) 


The vector xX; satisfies the eigenvector equation 


AK, = AX; (2.61) 
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If we now premultiply Eq. (2.61) by X; we obtain 

X; AX; = AjX;X; (2.62) 
If we now subtract Eq. (2.62) from Eq. (2.60), we have 

0 = (A; - A)X;X; (2.63) 
Since A, # Aj Eq. (2.63) gives the important relation 

x}X 7 = 0 (2.64) 
Since the scalar product vanishes, X; and X j are orthogonal. 

Property 2. If A is a real symmetrical matrix (a matrix all of whose 
elements are real numbers), all of its eigenvalues are real. This is not an 
obvious fact since the roots of an nth-order equation with real coeffi- 
cients are, in general, complex numbers. 

Let X be a typical eigenvector of A; therefore, we have 

AX = dX (2.65) 

The complex conjugate of this equation is 


AX* = d*X* (2.66) 


where the symbol * denotes the complex conjugate. The matrix A, 
being real, is unchanged. The transpose of Eq. (2.66) is 


XA ae, (2.67) 
We now postmultiply Eq. (2.67) by X to obtain 

X*AX = A*X*’X (2.68) 
But if we premultiply Eq. (2.65) by X*’ we have 


X*AX = dX*°X (2.69) 
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If we now subtract Eq. (2.69) from Eq. (2.68) we have 
Ore OF =x (2.70) 


However, if X consists of the elements x), x9, x3, ---, x, then X*’X is 
the sum of the squares of the absolute values of the elements of X. 
Hence X*’X is a nonzero positive quantity, and therefore to satisfy Eq. 
(2.70) we must have 


NPs = 2; (2.71) 


This equation implies that A is real. Since A is a typical eigenvalue, all the 
eigenvalues of A are real numbers. 

Property 3. If A is a real, skew-symmetrical matrix, so that A’ = —A, 
all its eigenvalues are imaginary. This may be established by the same 
argument that was used to establish property 2. 

Property 4. The eigenvalues of a real orthogonal matrix are all of 
unit modulus (have absolute value of unity). An orthogonal matrix has 
the property that A’A = I, that is, the inverse of an orthogonal matrix 
is its transpose. Matrices of this type are of great importance in dynam- 
ics and other branches of applied mathematics. Let A be an orthogonal 
matrix; its eigenvector equation is 


AX = 2X (2.72) 


Although A has real elements, A and X are not necessarily real. The 
conjugate of this equation is 


AX* = \*xX* (2.73) 
where A* denotes the conjugate of A. The transpose of Eq. (2.73) is 

X*A’ = d*x*” (2.74) 
If we now postmultiply Eq. (2.74) by AX = AX, we obtain 


X*’A‘AX = A*A(X**X) (2.75) 
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Since A is an orthogonal matrix A’A = I, Eq. (2.75) becomes 
X*’X = X*A(X*’X) (2.76) 
If we now divide by the scalar X*’X , we obtain 
MrA = 1 (2.77) 


It is therefore evident that the modulus (absolute value) of A is unity. 
Since d is a typical eigenvalue of the orthogonal matrix A, all its eigen- 
values are of unit modulus. 

The Transform of a Square Matrix (Similarity Transformations). In 
many applications of matrix algebra and matrix calculus to problems in 
physics and engineering, it is important to consider the matrix transfor- 
mation 


C= B71 AB (2.78) 


where B is a nonsingular matrix. Such a transformation is called a 
similarity transformation. The matrix C is said to be the transform of A 
by the matrix B. It is interesting to note that if A is a symmetric matrix, 
(A’ = A) and B is an orthogonal matrix (B~! = B’), then the transform 
of A, Cis 


C = BAB (2.79) 
In this case we have by the reversal law of transposed products: 
C’ = (B‘AB)’ = BAB = C (2.80) 
Therefore, since C’ = C,C is a symmetrical matrix. 
Property 5. The eigenvalues of C = B-! AB are equal to the eigen- 
values of A. That is, the eigenvalues of the transform matrix C are equal 
to the eigenvalues of A. To prove this proposition, we note that the 


characteristic matrix of C is given by 


(G =\DI=\(B LAB Ai =) BLA = DB (2.81) 
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Therefore the characteristic polynomial of C, Q(A) is given by 
Q(A) = det (C — Al) = det B-!(det (A — Al) det B (2.82) 


since (det B~!) = 1/det (B), it is apparent from Eq. (2.82) that 
QA) = det (A - AI) = pia) (2.83) 


where p(A) is the characteristic polynomial of A. Thus, it is evident that 
the matrices A and C have the same characteristic equations 


QA) = pir) = 0 (2.84) 


Therefore, the eigenvalues of C are equal to the eigenvalues of A. 

Property 6. If C is the transform of A so that C = B-1 AB, the 
determinant of C is equal to the determinant of A and the trace of C is 
equal to the trace of A. 

To prove this, it is only necessary to recall that the eigenvalues of A 
and C are the same. Since the determinant of a square matrix is equal 
to the product of its eigenvalues and the trace of a square matrix is 
equal to the sum of its eigenvalues, it is evident that A and C have the 
same determinant and the same trace. 

Property 7. The reciprocals of the eigenvalues of A are the eigenvalues 
of Av! and the eigenvectors of A are the eigenvectors of A!. To prove 
this statement, consider the eigenvector equation of A 


AX = dX (2.85) 
Let us premultiply Eq. (2.85) by A7 to obtain 
A1AX = IX = AA X (2.86) 
This equation can be written in the form 
Alx =a 1X (2.87) 
Equation (2.87) is the eigenvector equation of A’; therefore, A“ is the 


eigenvalue of A and X is the eigenvector of A+. Since and X are 
typical eigenvalues and eigenvectors, it is easy to see that the reciprocals 
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of the eigenvalues of A are the eigenvalues of A™!, and that the eigen- 
vectors of A are the eigenvectors of A. 


THE CASE OF REPEATED EIGENVALUES 


In certain practical cases it happens that the characteristic equation 
(2.22) has multiple roots; in such a case the matrix A is said to have 
multiple or repeated eigenvalues. The question of determining the 
eigenvectors that belong to a repeated eigenvalue is a difficult one. We 
shall consider here only the simplest aspects of this question. In order 
to make matters definite, we shall consider only a third-order case and 
then make the proper generalizations to cases of higher order. 

Let A be a (3, 3) matrix with a triple eigenvalue A). Let Xj, X,, and 
X, be the eigenvectors that belong to the eigenvalue A). The eigenvector 
equation for the eigenvalue Ap is written as 


7. epee va) 

(2.88) 
ANS = Ag Xe 
[Se a vor 


We can multiply the first of the equations (2.88) by a, the second one 
by b, and the third one by c, add the results, and thus obtain 


AlaX, + bXy + cX3] = AplaX, + bXq + cXg) (2.89) 
If we now let 

Y = (aX; + bX, + cX3) (2.90) 
it is apparent that Eq. (2.89) can be written in the form 

AY = AY (2.91) 
Therefore, Y is an eigenvector of A belonging to the eigenvalue A,; hence 
a linear combination of the eigenvectors belonging to the multiple eigen- 


value Ay is also an eigenvector belonging to Aj. This fact can be utilized 
in obtaining eigenvectors belonging to multiple eigenvalues. 
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Example. As an example of a symmetrical matrix that has a double 
eigenvalue, consider the matrix 


[i Si ae | 
Az=|1 0 =1 (2.92) 
1-1 0 


This matrix has the following eigenvalues: 


Ae ina VS ad tk (2.93) 


eee etl (2.94) 


eT Da xy 
1 0 -1]]x,] =| x2 (2.95) 
1-1 O X3 


This equation if expanded yields three equations of the type 
X%3 = (xy — Xq) (2.96) 
We can therefore take 


ng =| (2.97) 


for the eigenvector belonging to the eigenvalue dg = 1. 
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In order to obtain the eigenvector X, belonging to A; = 1 we note 
that the matrix A is a symmetrical matrix and that as a consequence of 
property 1 above, the eigenvectors of A are mutually orthogonal. Let us 
make use of Eq. (2.96) in the form 


Xy = Xp + Xg (2.98) 


and take x, = h and xg = k so that the third eigenvector has the form 


Xg = h (2.99) 


We now adjust X, so that 

X{X3 = -(h+ bh +h+k=0 (2.100) 
and 

XjXg = (h+b+h=0 or k= -2h (h+h) = -h (2.101) 


If we substitute these values of k into Eq. (2.99), we obtain 


—h -1 
X, = hj=hj 1 h = an arbitrary constant (2.102) 
-2h -2 


If we let h = —1, we finally obtain 


aii 1 1 
came leila ee [ele ee aed | 2 (2.103) 
1 0 2 


as a set of three orthogonal eigenvectors of the matrix A. 
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Example. Let it be required to find the eigenvalues and the eigen- 
vectors of the matrix 


7 #4 -4 
A=| 4 -8 -1 (2.104) 
-4 -1 -8 


NOE PCT, ee eur aye eat (2.105) 


The eigenvector X, can be found by the standard procedure of Eq. 
(2.52) to be 


X, =] 1 (2.106) 


When A, = Ag = -9, the eigenvector equation will be found to give three 
equations all proportional to the equation 


4%, +%_2-%3 = 0 or 4x, = xg - Xx (2.107) 


If we choose X= 1, x3 = 1, then x, = 0; therefore, we can take X, to be 


0 


Telia (2.108) 
1 


If we now take X_ = h, xg = k, then x, = (k - h)/4, and we can therefore 
take 
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4 (2.109) 


Since A is symmetric, X,, Xj, and X are orthogonal, therefore Xj X3 = 
0, X5 X,=0. We find that k =—h ensures orthogonality, and if we 
choose k = 2, we have 


5 Ca teers (2.110) 


4 0 1 
X, = 1 Xe =| 2 X, =|-2 (2.111) 
=i 1 2 


MATRIX ITERATION METHOD OF 
EVALUATING EIGENVALUES AND 
EIGENVECTORS 
As discussed above, the characteristic values and vectors of matrix [A] 
are related as follows: 


[A - al](S) = 0 (2.112) 


This matrix equation can be written 
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a, -A ay @)3 --- G4, 8} 0 
991 Sas pycna es palsy (2.113) 
any a9 43 +--+ G,-A||8, 0 


where A is the characteristic value, a’s are the elements of matrix [A], 
and s’s are the elements of the characteristic vector corresponding to A. 
It is further noted that in many engineering applications the matrix A 
is a real-valued symmetrical matrix, that is, a;, = aj; and a;,’s are real. 
From matrix theory discussed above it is known that the characteristic 
values of real symmetrical matrices are real. This property will be 
utilized in obtaining characteristic values and vectors by the following 
iteration process. 
Equation (2.113) can also be written in the form 


tea. “O18, Mp, 1) $1 Sy 
Ce 3 ny mu Ie El fF -8| (P r a2 (2.114) 
Gre Onde dee = Saran Sn 


In the matrix iteration method of finding the value of \ and the corre- 
sponding values of sj, s9,...,s, Wwe start with an arbitrary set of 
$1, Sg, --+, 8,, and for convenience in calculations choose s, = 1. (It 
should be noted that Eq. (2.113) forms a set of homogeneous equations, 
i.e, equations involving s’s with the right side equal to zero. The 
absolute values of s’s cannot be determined. However, ratios of s’s can 
be obtained. By setting s,, = 1,in effect we are calculating the ratios of s’s 
tos, OF 81/8, 89/8, 83/8, +--+, S,/S,,.) Setting these values of s’s on the 
left of Eq. (2.113) and multiplying by the matrix [A], we obtain a 
column matrix. Dividing this column matrix by the element in the last 
TOW s,,, we obtain the next approximation to the value of \(= s,) and 
the characteristic vector consisting of sj, so, ..., s,- Continuing this 
process the iteration will converge, resulting in the characteristic value A 
and the corresponding characteristic vector [4]. It can be shown that 
the characteristic value obtained by this method is the largest characteristic 
value or equivalently the largest root of the characteristic equation. 
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It can be shown that the rate of convergence of this iteration process 
depends on the numerical separation of the eigenvalues of matrix A. 

Example. As a numerical example of evaluating characteristic values 
and vectors by the matrix iteration method, consider the following 
matrix: 


2-1 0 
-1 2 -1 (2.115) 
0-1 1 


As a first approximation to s’s or the characteristic vector, consider the 
column matrix 


= (2.116) 


Note that the last element is set to unity. It is also generally known that 
the elements of the characteristic vector corresponding to the highest 
characteristic value are not all of the same sign. For this reason the sign 
of the center element is chosen as minus. The iteration method will, of 
course, converge even if [1, 1,1] were chosen. However, in this case 
more iterations will be required. Multiplying the two matrices and 
keeping the last element of the resulting column matrix as unity, we get 


2-1 0 1 1.5 
<b) V2 SDS 1S 2 =e (2.117) 
0-1 #1 1 1 


Using the column matrix on the right as the next approximation and 
following the above procedure, we get 
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2). <1. 0 1.5 1.6666 
=i Qt —2 = 3 — 2.1666 
6 -l 1 1 1 
1.6666 L7. 
a (2.118) 
same as above —2.1666 | = 3.1666] -—2.2105 
1 1 
1.7368 1.7701 
same as above | |~—2.2105| = 3.2105]-2.2295 
1 1 
Continuing this process, iterations 10 and 11 will result in 
1.802 1.802 
3.247 |-2.247] and 3.247 |-2.247 (2.119) 
1.0 1.0 


It is seen that iterations converge and the values at iterations 10 or 11 
represent the largest characteristic value (3.247) and the corresponding 
vector 1.802, —2.247, 1.0. It is sometimes desirable to normalize the 
characteristic vector to a value of unity. If $1, Sg, and sg are the 
components of the characteristic vector, this results: 


Sy" “ eae + ee =a (2.120) 


Or in the case of the above example, all the components should be 
divided by the square root of 


(1.802)? + (2.247)? + (1.0)? = 9.2962 (2.121) 


or 3.049. The normalized characteristic vector then becomes 
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ple, 0.591 
3.049 
y! 
ae (2.122) 
3.049 
BLUE 0.328 
3.049 


FORTRAN PROGRAM FOR EVALUATION 
OF CHARACTERISTIC VALUES AND 
VECTORS 


The following FORTRAN program [1] has been written for the 
iteration process discussed above. The data entered in the program are 
the same as the above example. The output of the program consists of 
20 iterations and is given following the listing of the program. 

Statement 10 reserves computer storage for indexed variables. 
Statements 20 and 30 read the order of the matrix (N = 3) and 
the values of its elements S(I,J). Statement 40 reads the initial 
characteristic vector P(J) as (1, —1, 1). K in statement 50 is the 
number of the iteration and is indexed by one at each iteration. 
Statements 70 through 110 multiply the matrix by the initial char- 
acteristic vector. Statements 120 through 140 change the desig- 
nation of the characteristic vector from Z(I) to P(I) to be used in 
future iterations. Statements 150 through 180 print out the iteration 
number and the characteristic value which is equal to P(N). State- 
ments 190 through 210 divide the characteristic vector by P(N), 
as was done in the example problem, to keep the last element of 
this vector equal to unity. Statements 220 and 230 print out the 
characteristic vector. In statement 240 the number of iterations is 
arbitrarily set at 20. If K is less than or equal to 20, the loop starting 
with statement 60 is repeated and another iteration is obtained. 
The computer results are shown following the program. It is seen 
that the characteristic vector and the corresponding characteristic 
value to three significant figures remain unchanged after iteration 
number 10. 
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c PROGRAM TO EVALUATE CHARACTERISTIC VALUE AND VECTOR 
10 DIMENSION $(100,100),P(100),2(100) 
20 DATAN/3/ 
30 DATA ((S(I,J),J=1,3),1=1,3)/2.,-1.,0.-1.,2.-1.,0.,-1.,1./ 
40 DATA (P(J),J=1,3)/1.,-1.,1./ 
50 K=0 
60 K=K+1 
70 DO 110 I=1,N 
80 Z (I)=0. 
90 DO 110J=1,N 
100 Z(1)=S(1,J)*P(J)+Zl) 
110 CONTINUE 
120 DO 140 I=1,N 
130 P(I)=Z(I) 
140 CONTINUE 
150 WRITE (2,160) K 
160 FORMAT (18H ITERATION NUMBER= 15) 
170 WRITE (2,180) P(N) 
180 FORMAT (22H CHARACTERISTIC VALUE= 1PF10.3) 
190 DO 2101=1,N 
200 P(!)=P(1)/P(N) 
210 CONTINUE 
220 WRITE (2,230) (P(!), 1=1,3) 
230 FORMAT (22H CHARACTERISTIC VECTOR /(6X1PE10.3)) 
240 IF (K-20) 60,250,250 
250 STOP 
260 END 


Computer solution obtained: 


ITERATION NUMBER = 1 
CHARACTERISTIC VALUE = 2.000E 00 
CHARACTERISTIC VECTOR 
1.500E 00 
—2.000E 00 
1.000E 00 
ITERATION NUMBER = 2 
CHARACTERISTIC VALUE = 3.000E 00 
CHARACTERISTIC VECTOR 
1.667E 00 
—2.211E 00 
1.000E 00 
ITERATION NUMBER = 3 
CHARACTERISTIC VALUE = 3.167E 00 
CHARACTERISTIC VECTOR 


1.737E 00 
—2.211E 00 
1,000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.770E 00 
—2.230E 00 
1.000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.787E 00 
—2.239E 00 
1,000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.795E 00 
—2.243E 00 
1.000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.798E 00 
—2.245E 00 
1.000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.800E 00 
—2.246E 00 
1.000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.801E 00 
—2.247E 00 
1.000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.802E 00 
—2,247E 00 
1.000E 00 
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10 


3.211E 00 


3.230E 00 


3.239E 00 


3.243E 00 


3.245E 00 


3.246E 00 


3.247E 00 
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ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.802E 00 
—2.247E 00 
1.000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.802E 00 
~2.247E 00 
1.000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.802E 00 
—2.247E 00 
1.000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.802E 00 
—2.247E 00 
1,000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.802E 00 
—2.247E 00 
1,000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.802E 00 
—2.247E 00 
1.000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 
1.802E 00 
—2.247E 00 
1.000E 00 
ITERATION NUMBER = 
CHARACTERISTIC VALUE = 
CHARACTERISTIC VECTOR 


11 


12 


13 


14 


15 


16 


a? 


18 


3.247E 00 


3.247E 00 


3.247E 00 


3.247E 00 


3.247E 00 


3.247E 00 


3.247E 00 


3.247E 00 
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1.802E 00 
—2.247E 00 
1,000E 00 
ITERATION NUMBER = 19 
CHARACTERISTIC VALUE = 3.247E 00 
CHARACTERISTIC VECTOR 
1.802E 00 
—2.247E 00 
1.000E 00 
ITERATION NUMBER = 20 
CHARACTERISTIC VALUE = 3.247E 00 
CHARACTERISTIC VECTOR 
1,802E 00 
—2.247E 00 
1.000E 00 


ITERATION METHOD FOR OBTAINING 
THE SMALLEST CHARACTERISTIC 
VALUES AND VECTORS 


The method described above gives the highest characteristic value and 
the corresponding characteristic vector. However, in many applications 
the lowest characteristic value and the corresponding vector are desired. 
To obtain this, we divide Eq. (2.114) by A and premultiply both sides by 
matrix (a, Denoting 1/A = w, we get 


4 Sr Fine Sig 17 ey 
leet cet, 228 ager tan oe) (2.123) 

SH an) q,2 413 Ian Sp 
We note that the iteration process for obtaining and sj, S9, $3, ---, S, 


remains the same as before with matrix a;,] replaced by its inverse 
a," as given by Eq. (2.123). From this equation the value of » will 
correspond to the largest characteristic value of the inverse matrix [a; at 
or the smallest characteristic value of [a ijl since = I/A. 

Example. The inverse of the matrix of the previous example can be 
obtained, for example by augmented matrix method, as written below 
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Using this inverse together with an initial iteration characteristic vector 


and proceeding with the iteration processs as was done in the above 
example, we get 


First iteration 


Ae Dh cieinad 0.50 
3 Meet te bl it = 6 0.8333 
A: Sue 1.0 


1 1 1] 0.4451 0.4451 
1 2 21] 0.8020) = 5.049] 0.8020 


tJ 2° SRO 1.0 


1. 1. 1] | 0.4450 0.4450 
1 2 2] {0.8019} = 5.049] 0.8019 
eee lel (AY) 1.0 
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From this, the smallest characteristic value will be A = 1/5.049 = 
0.198 with the corresponding components of the characteristic vector 
0.445, 0.8019, and 1.0. 

This vector can be put in normalized form by dividing the components 
by the square root of the sum of the squares. This results in the values 
0.328, 0.591, and 0.736 (normalized) with the lowest characteristic value 
of A= 0.198. 


USE OF ORTHOGONALITY RELATIONS IN 
OBTAINING CHARACTERISTIC VALUES 
AND VECTORS 


After obtaining the largest characteristic value and the corresponding 
characteristic vector it is possible to use the orthogonality relations to 
obtain other characteristic values and vectors. (This method is also used 
when the characteristic equation has double roots, i.e., two characteristic 
values are identical.) From matrix theory we have that the characteristic 
vectors of real symmetric matrices are orthogonal. That is, if 
S81, S9)-++» 8, are the components of the characteristic vector corre- 
sponding to the characteristic value \, and sj, s5, s3,.-., 8; are the 
components of the characteristic vector corresponding to \’, then 


818] + S989 + -+: + 8,8), = 0 (2.124) 


Assuming that we have obtained the largest characteristic value A, and the 


corresponding vector sj, So, .-., s,, We can obtain the next characteristic 
value and vector )’ and sj, s9, --., s;, by writing Eqs. (2.114) in the form 
as | Ramee Rl fa au 


pes atcierae sn aise ‘ = (2.125) 
a1 a9 Inn Sn Sn 
Solving Eq. (2.124) for s/,, we get 
; St | Sota. oe era, 
8 = - 9) -2-25j----2 ts, (2.126) 
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Writing matrix equation (2.125) in the form of linear equations and 
substituting the value of s/, from Eq. (2.126), we get a set of n equations 
in(n — 1) unknowns 8} 8g; +++. 8) 4+ 

Since the last equation is redundant (it will always be satisfied) we can 
drop this equation and write 


8] s} 
se | _ iss (2.127) 
Sn=1 8-1 


where b’s are the new coefficients. These equations can be solved by 
the iteration process, discussed previously, for \’ and sj, s5, ---, § 
Having these values, Eq. (2.126) will give the value of s;,. 

Additional characteristic values and vectors can be obtained by writing 
the orthogonality relations between the vectors already calculated and 
vectors whose values are desired, and using these relations to reduce the 
number of equations. 

Example. Consider the previously given example problem where the 
largest characteristic value and the corresponding vector of the matrix 


n=1° 


2-1 0 
-1 2 -1 
0-1 1 


were found by iteration method. These values are 3.247 and 1.802, 
—2.247, 1.0. It is desired, using the orthogonality condition, to find the 
next largest characteristic value A’ and the corresponding vector with 
components sj, s5 and s3. 

The orthogonality condition of Eq. (2.126) results in 


83 = -—s,-— 83 


as ees (2.128) 


= -1.802s; + 2.24785 
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The values of sj, s5,and s3 should also satisfy Eq. (2.125): 


2-1 0 s} Ss} 
-1 2 -1] | sg] = A’]s9 (2.129) 
0 -1 1) I{s3 83 


Writing the above relation in equation form and substituting for s3 in 
terms of sj and sq in the first two of these equations, we get 


(2 - As} - sp = 0 
0.8025; + (-0.247 - X85 = 0 NED) 


-8_ + (1 - A’)s3 = 0 


The last equation of (2.130) is not needed as the first two equations are 
sufficient to solve for s}, s5,and A’. Putting these in matrix form, we get 


2 -1 Ss} 8} 
= (2.131) 
0.802 -0.247] | sp 85 


The above relations, which correspond to Eq. (2.127), can be solved 
by matrix iteration,* discussed previously, by assuming that s; = 1.0 
and sg = 1.0 for the first iteration. The iteration process will converge 
to three significant figures at the sixth iteration, resulting in 


WV = 1.555 
sj = 2.247 (2.132) 
85 = 1.0 


Using the above values in the expression for s3 in terms of sj and s3, 
we get 


*Since in this case only two equations are involved, they can also be solved 
simultaneously. 
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83 = -1.802s; + 2.24785 
= -1.802 x 2.247 + 2.247 x 1.0 
= -1.802 


(2.133) 


The value of s3 can be set equal to 1.0, as was done in the case of highest 
characteristic value and vector, by dividing the components of the new 
characteristic vector by —1.802. This results in 


8, = = -1.247 
— 1.802 
1.0 
85 = = —0.555 2.134 
2 ae ie ) 
a4 = — 1.802 Sark 
— 1.802 


with A’ = 1.555. The above values can be normalized by dividing each 
component by the square root of the sum of the squares. This results in 


8; = -0.736 s5 = -0.328 s3 = 0.591 


In order to calculate the next characteristic value and the corresponding 
vector we can proceed as follows. Assume the desired values are \” and 
Sj, 89, and sg. Write the orthogonality relations between these and the 
vectors S and S‘as given in Eq. (2.124). 


1.8028] - 2.24783 + sj = 0 
» - (2.135) 
-1.247s{ - 0.55585 + s3 = 0 
Solving these equations for s3 and s5 in terms of sj, we get 
sg = 1.802s{ 
” (2.136) 
83 = 2.24787 


Substituting the above relations for the values of s3 and sg in the first 
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of Eqs. (2.129) written for s”, 


Pipe ih FFE 
Nye) Elias | as] 85 (2.137) 


we get 

2sj - sg + 083 = A”s{ (2.138) 
or 

2sj - 1.802s{ = A”s{ (2.139) 


Setting the value of sj in the above equation equal to unity, we get 
d” = 0.198 (2.140) 
And, from the previously given equations, 


sj = 1.802 


os setae (2.141) 
83 = & 


The above characteristic vector can be written in the form where s3 = 
1.0 by dividing the components of the vector by sz = 2.247. This 
results in 


st = 0.445 sz = 0.802 sg = 10 


These values compare with previously obtained results for the lowest 
characteristic value. The normalized form of this characteristic vector 
can be obtained by dividing the above values by the square root of the 
sum of the squares. This gives 


sj = 0.328 sg = 0.591 3 = 0.736 (2.142) 
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An additional method for the computation of the characteristic 
equations and the corresponding vectors will be discussed in the fol- 
lowing section. This method, referred to as the method of A. M. 
Danilevsky, is based primarily on the properties of similarity transfor- 
mation discussed previously. 


THE METHOD OF A. M. DANILEVSKY FOR 
OBTAINING THE CHARACTERISTIC EQUATION 


The method of Danilevsky for obtaining the characteristic equation is 
based on the reduction of matrix A, whose characteristic equation is 
desired, 


yf aed Ve tera B) 7 
Aoesiin2l Sas Sag es Se, (2.143) 
Gn1 Gna Sng Gan 


1 0 0 0 
P=1/0 derszenh 0 (2.144) 
0 Oe. 1 0 
where the elements pj, po, pg, ---, p,, Tepresent the coefficients of the 


characteristic equation of matrix A. 

It can be shown that the transformation of matrix A to matrix P can 
be accomplished by a series of similarity transformations. Thus, on the 
strength of the theory that the characteristic equations of similar matrices 
are identical, the characteristic equations of matrices A and P will be the 
same. The fact that the elements of matrix P represent the coefficients 
of the characteristic polynomial is readily seen by writing the characteristic 
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determinant of P as 


Py -A Pg Pg +++ Ph 


1 -A 0... 0 
P(A) = 0 1 SAS cee (2.145) 
0 0 0 ...-A 


and expanding it in terms of the elements of the first row 
D® = 1a" - pat} - «+. -p,) = 0 (2.146) 


where D(A) is the desired characteristic polynomial and n is the order of 
the square matrix A. 

We demonstrate the transformation from matrix A to matrix P for a 
4 = 4 matrix 


Myotis 1a) 94 
i ae aa = a (2.147) 
ce a a 


yy, Sa a ay 


Postmultiplying Eq. (2.147) by the matrix M, which is formed from the 
elements of the fourth row of the matrix A, shown below 


1 0 0 0 
0 1 0 0 
Men) Aaa" tap 1 444 (2.148) 
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will result in 


nw (2.149) 


where the elements b of the above are 


913% 913%,42 Ti3 
bya = 91 - bin = G2 - > bg = —,--- 
43 943 943 
993741 b 993949 
Paty RiGnl icy pate av Slee? 
43 43 


(2.150) 

Note that the last row of the matrix of Eq. (2.149) is the same as that of 

the P matrix of Eq. (2.144). The matrix AM of Eq. (2.149) is not as yet 

similar to matrix A of Eq. (2.147). It can be made similar to A by 

premultiplying by the inverse of matrix M. Thus, the similarity transfor- 
mation of A into the form of Eq. (2.149) is written as 

C = M-1AM (2.151) 


where M~ is readily obtained from Eq. (2.148) as 


0 1 0 0 (2.152) 


The third row of this matrix is the same as the last row of the matrix 
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of Eq. (2.147). The premultiplication of matrix AM of Eq. (2.149) by 


M71, Eq. (2.151), does not affect the last row of the matrix of Eq. 
(2.149). 


At this point the matrix C of Eq. (2.151) will be of the form 


(2.153) 


Suis “19, “1g 1 


Gay az) “San Cea iGov So3, Sag: iCke 


Say Cag) Cag ea 


Oe CO) Ota | (OR Ones 


0 
(2.154) 

cies 0 0 0 dy, yo dyg ayy 

©s1) ae <Css. S44. 

= Sa dy, dyg dy3 dog 

C39 gp Cap 32} _ 

0 0 1 0 0 1 0 0 
OO ale oa Or 200 L850 


Carrying the above process a step further will reduce the 4 x 4 matrix 
of Eq. (2.147) to the form of Eq. (2.144), that is, 


Py Pe Bg 


1 0 0 0 (2.155) 
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where p’s represents the coefficients of the characteristic polynomial 
of Eq, (2.146). 
As an example of the above procedure, consider the 4 x 4 matrix 


—5.50988 1.87009 0.422908 0.008814 
0.287865 -11.8117 5.7119 0.058717 (2.156) 
0.049099 4.30803  -12.9707 0.229326 
0.006235 0.269851 1.39737  - 17.5962 


whose characteristic equation is required. Following the above procedure 
the matrix M of the similarity transformation is calculated as 


iL 0 0 0 
a 0 1 0 0 (2.157) 
-0.00462 -0.193113 0.71563 12.5924 
0 0 0 1 


The matrix M* is calculated as given by Eq. (2.152). That is, 


1 0 0 0 
rer ae 1 0 0 (2.158) 
0.006235 0.269851 1.30737 - 17.5962 
0 0 0 1 


Having matrices M and M!, we implement the similarity transformation 
of Eq. (2.151) 
—5.51177 1.78842 0.302646 5.33423 
0.262379 -12.9147 4.08761 71.9851 
0.185919 6.04616 —29.462 — 208.456 


0 0 1 0 
(2.159) 


Eigenvalues, Eigenvectors, and Quadratic Forms 93 


At this point we set matrix C above identical to matrix A and repeat 
the similarity transformation about the third row of the above matrix. 
The M matrix for this case becomes 


at 0 0 0 
—0.03075 0.165394 4.87284 34.4774 (2.160) 

> 0 0 1 0 

0 0 0 1 


The inverse of the M matrix can be obtained from the third row of the 
C matrix as 


1 0 0 0 

Si 0.185919 6.04616 -29.462 -208.456 (2.161) 
0 0 1 0 
0 0 0 1 


The resulting matrix after the similarity transformation becomes 


—5.56676 0.295794 9.01733 66.9944 
2.95252 -42.3217 — 562.559 — 2244.47 
0 1 0 0 


M-lCM = 


0 0 1 0 (2.162) 


We can set the above matrix identical to A and repeat the procedure. 
This, after obtaining matrices M,M7!, and M-!cM, will result in the 
matrix of the coefficients 


— 47.8885 -—797.281 -5349.48 -12296.6 
1 0 0 0 (2.163) 
0 1 0 0 
0 0 1 0 


94 Digital Computer Methods in Engineering 


From the above matrix and Eq. (2.146), the characteristic equation of 
the problem can be written 


Dd) = (- D404 + 47.8885A3 + 797.281A2 
. Se (2.164) 


+ 5349.48A + 12296.6) = 0 


The roots of this equation can be obtained by the methods described in 
Chap. 3. 


COMPUTER PROGRAM FOR THE METHOD OF 
A.M. DANILEVSKY 


The following computer program has been written in BASIC program- 
ming language (see [8]). This programming language includes matrix 
operations denoted by MAT at the start of each statement. The program 
starts with dimension statement 100 reserving space for matrices A, M, I 
(inverse of M), and P = AM. Statement 110 specified the order of the 
matrix whose characteristic equation is required. Note that in the 
BASIC language the first row and first columns of a matrix are denoted 
by row zero and column zero, respectively. For this reason N = 3 
specifies a fourth-order matrix. Statement 120 reads the elements of 
matrix A, from left to right, as given in data statements 350 through 
380. Statements 160 through 180 set matrices M,/, and P equal to 
identity matrix, zero matrix, and zero matrix, respectively. Statements 
190 through 220 form the similarity transformation matrix M. State- 
ment 250 obtains! = M7! matrix. Statement 280 obtains the P = AM 
matrix while statement 290 calculates M"! AM matrix and denotes this 
as the A matrix for the second iteration which starts from statement 
number 160 and so on. 

The results given for the example problem were obtained through the 
above described computer program. 

In exceptional cases the process of similarity transformation, as 
described above, may result in a matrix where a zero may appear in 
the position of an element whose value should be transformed to unity 
in the next iteration. Consider the C = M! AM matrix at the end of 
some iteration to have the form 


100 
110 
120 
130 
140 
150 
160 
170 
180 
190 
200 
210 
220 
230 
240 
250 
260 
270 
280 
290 
300 
310 
320 
330 
340 
350 
360 
370 
380 
390 
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DIM A (10,10),M(10,10),1(10,10),P(10,10) 
LET N=3 

MAT READ A\(N,N) 

PRINT “MATRIX A” 

MAT PRINT A 

LET L=N-1 

MAT M=IDN (N,N) 

MAT I=ZER (N,N) 

MAT P=ZER (N,N) 

FOR J=0 TON 

LET M(L,J)=—A(L+1,J)/A(L+1,L) 

NEXT J 

LET M(L,L)=1/A(L+1,L) 

PRINT “MATRIX M” 

MAT PRINT M 

MAT I=INV (M) 

PRINT “INVERSE OF MATRIX M” 

MAT PRINT | 

MAT P=A*M 

MAT A=1*P 

PRINT “---------- MATRIX A---------- Mi 
MAT PRINT A 

LET L=L-1 

IF L<O THEN 390 

GO TO 160 

DATA —5.50988, 1.87009, .422908, .008814 
DATA .287865, —11.8117, 5.7119, .058717 
DATA .049099, 4.30803, —12.9707, .229326 
DATA .006235, .269851, 1.39737, -17.5962 
END 


C= 
0 0 1 0 0 
0 0 0 1 0 
Resign eis nisisir ce btarnne wie eeraoe 1 


(2.165) 


with the element c, ,_; = 0. In this case a nonzero element is chosen 
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along the same row, c,;(i < k — 1)and the ith and k — 1 columns and 
simultaneously the ith and k — 1 rows are interchanged. This operation 
will be equivalent to a similarity transformation with the M matrix 


1 
vil 
elma gl ith row 
jl | 
Mie ee (2.166) 
| | 
Nei) k — lrow 
jl 
ith | 
col. | ea 
col. 


The above matrix has the property M? = I, or M = M~!; therefore, post- 
multiplication and premultiplication of matrix C of Eq. (2.165) by M 
and M~+ respectively, will represent a similarity transformation with no 
change to the characteristic equation. 


CHARACTERISTIC VECTOR BY THE METHOD 
OF A. M. DANILEVSKY 


Let A be one of the characteristic values of matrix A obtained from 
the characteristic polynomial derived above. The characteristic vector 
corresponding to A is denoted by X = (xj, x9, x3, ---, x,) where small 
x’s are the components of the vector. The characteristic vector corre- 
sponding to the matrix P is denoted by Y = (¥45 Yor +9 Wy) It can be 
shown that 


X = M,My ...M,_1Y (2.167) 


Ease: 
where M,,Mg,...,M,_ 1 are, respectively, the transformations which 
reduce the matrix A of Eq. (2.143) to matrix P of Eq. (2.144). The 
components of the characteristics vector of Eq. (2.144) can readily be 
obtained from 
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Brees Po. Ds P| 1 
1 ASD eet Onilliling 
0 I) ek o | |y,}= 9 
0 0 Oo =X Ws 


which in equation form becomes 


y= Apoy + Pade +2 + Pain = 0 
¥y - We = 0 


yey — Wh = 0 


Setting y, = 1 we get ¥,-1 = 4, %,-2 = a, y, - 7-1. Having these 
values and the product of M transformations, Eq. (2.167) will result in 
the characteristic vector corresponding to A. 

At this point a comparison of the previously discussed matrix iteration 
method and the method of A. M. Danilevsky is in order. In the method 
of Danilevsky, the characteristic equation is obtained and then the roots 
of this equation, the characteristic values, are calculated by numerical 
root finding methods of Chap. 3. Thus, the numerical proximity of the 
roots does not increase the number of required calculations as was the 
case in matrix iteration method. On the other hand, the matrix iteration 
method produces the largest eigenvalue and the corresponding eigen- 
vector at the same time. Since in many engineering problems the highest 
(or the lowest, which can be obtained by the inversion of the matrix of 
the coefficients) eigenvalue and the corresponding eigenvector of systems 
represented by real symmetrical matrices are required, the matrix iteration 
method under these conditions is preferred. 

The number of multiplications and divisions by the method of A. M. 
Danilevsky in obtaining the characteristic equation is (n - D(n2 42-1) 
where n is the order of the matrix. The method of A. N. Krylov for 
obtaining the characteristic polynomial requires a greater number of 
arithmetical operations than Danilevsky’s method. It is used primarily 
because of the simplicity of computational procedure. Another short- 
coming of Krylov’s method comes from the fact that arithmetical 


98 Digital Computer Methods in Engineering 


operations are to be performed on numbers of different orders of 
magnitude, resulting, in many cases, in loss of accuracy. In cases where 
the characteristic equation has multiple roots, Krylov’s method results 
in the coefficients of lower-degree polynomial. 

The iteration methods of calculating eigenvectors and eigenvalues 
discussed above were primarily given from a computational point of 
view. Characteristic values and vectors, of course, have definite physical 
meanings in vibration problems as described in the following pages. 


CHARACTERISTIC VALUES AND VECTORS IN 
MECHANICAL VIBRATIONS ~ 


It will be shown in the following example that the characteristic 
values correspond to the natural frequencies of vibration in mechanical 
systems while the characteristic vectors correspond to the normal modes 
of vibration. If a given conservative (no friction) dynamical system is 
initially displaced proportional to one of its normal modes, the system 
will vibrate indefinitely with a frequency equal to the natural frequency 
of the system corresponding to that normal mode. 

Consider the dynamical system of Fig. 2.1. The moment of inertia of 
each disk is denoted by | while the torsional stiffness of the connecting 
shaft is denoted by c. The length of the shaft between disks is 7. The 
equations of motion of each disk from Newton’s second law of motion 
can be written 


Sie <¢ +2 q = o) 

1 I 1 1 1 2 
Wig = “(ay ~ 99) + “(ag - a9) (2.168) 
-li, = 5% =053) 


ies oupene stot 


figure 2-1 Vibrating shaft with three disks. 
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where q’s are the angular displacements and dots represent derivatives 
with respect to time. The above equations can also be written in the form 


lif, = = (ay Sida) 


-IGy 


c 

Fes 249 - 95) (2.169) 
-lis = = (dg + 3) 

Consider the solution of the above equations in the form 


q; = A;sinfot +p) i = 1,2,3 (2.170) 


Obtaining the second derivative of the above equation and substituting 
in Eq. (2.169), we obtain 
("29a 
c 


(20) 4, (2.171) 


e 


Il 
= Alt Ag = (Ho?) A, 


¢ 


2A, - Ag 


ATH edg ait 


The above homogeneous equation can have nonzero solutions for A,, Ag, 
and A, only if the following determinant vanishes 


-1 2-2 -1 }=0 (2.172) 


A = —o (2.173) 
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With this definition of \ Eqs. (2.171) can be written in matrix notation 
as follows 


210i) [As Ay 
ai ||| aay (2.174) 
0 =1 1) [A, Ag 


The roots of the determinant of Eq. (2.172) were previously calculated 
to be 


A, = 3.238 Ay = 1.555 Ag = 0.198 (2.175) 


Using these values in Eq. (2.174) will result in corresponding values of 
A’s, which were also previously calculated as 


‘A, = 0.591 
AL = i = -0.736 
la, = 0.328 
A, = -0.736 
Ag = 1.555 <A = -0.328 Bec 
Az = 0.591 
A; = 0.328 
Ag = 0.198 <A, = 0.591 
[Ay = 0.736 


Note that the angular frequency can be calculated from the values of 
X’s using Eq. (2.173); thus 
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eee 8 (3 (2.177) 
z Il 
wg = 0.445 5 


Using the above values and the values A from Eq. (2.176) in Eq. (2.170), 
we obtain for the complete solution 


4, = 0.591A sin (, t+ ¥y) - 0.736B sin (wot + vy) 
+ 0.328C sin (wt + Ws) 


dg = -0.736A sin (w,t +H) — 0.328B sin (wot + 9) (2.178) 
+ 0.591€ sin (wat + vg) 

dg = 0.328A sin (wt + 441) + 0.591B sin (wot + vy) 
+0.736C sin (agt + v3) 


It should be remembered that the A’s of Eq. (2.176) represent the ratio 
of characteristic values rather than their absolute values. For this reason 
the constants A, B, and C are added to the above equations. 

In dynamics the values of o as given by Eq. (2.177) are referred to as 
the natural frequencies of vibration and corresponding values of A’s of 
Eq. (2.176) are modes of vibration. 

It can be shown that an initial displacement of the disks of Fig. 2-1 
proportional to the values of the modes of vibration (Eq. (2.176)) will 
result in oscillations of the system at the rate given by the corresponding 
frequency. 

To prove the above statement, assume the following initial conditions 
for the system of equations (2.178): 


att = 0 4g, = 0.591 go =-0.736 3 = 0.328 


4, =0 4 = 0 43 = 9 
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Using the derivative condition above in Eq. (2.178) we obtain the 
following set of equations 


0.591 -0.736 0.328 | | Aw, cos v, 
-0.736 -0.328 0.591] | Bas cos wy | - 0 (2.180) 
0.328 0.591 0.736 | | Cw cos wy 


Since the determinant of the coefficients of the above equation is not 
zero, the only way that the equation will be satisfied is 


cosy, = 0 vy, = 90° and vy = 90° wy = 90° (2.181) 


Using the initial condition for q,,q5, and 3 from Eqs. (2.179) and 
(2.178), we get 


0.591 -0.736 0.328] | A 0.591 
-0.736 -0.328 0.591] | B | = | -0.736 (2.182) 
0.328 0.591 0.736] | C 0.328 


From the above equation 
A=1 B=0 C=0 (2.183) 
And, the system of equations (2.178) becomes 


9; = 0.591 cos wt 


{2 = -0.736 cos w,t (2.184) 


93 = 0.328 cos wt 


Thus, the system will vibrate with the frequency @, once displaced by 
the normal modes corresponding to this frequency. 
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THE RELATION BETWEEN THE EIGENVECTORS 
AND EIGENVALUES OF A MATRIX AND THOSE 
OF ITS TRANSPOSE 


In this section we will consider the relation of the eigenvalues and 
eigenvectors of a matrix A and those of its transpose A. 
Let the eigenvector equation of the matrix A be 


AX = AX (2.185) 
Let the eigenvector equation of A’ be 
A'Y = pY (2.186) 
The characteristic equation of the matrix A is 
P(A) = det (A - AD = 0 (2.187) 
The characteristic equation of A’ is 
Q(A) = det (A’- pl) = 0 (2.188) 


However, since interchanging the rows and the columns of a determi- 
nant does not affect its value, we have 


P(A) = det (A — Al) = det (A - AI)’ = det (A’- AD (2.189) 
we see that the form of the characteristic equation of A is the same as 
the characteristic equation of A’ and therefore that A and A’ have the 
same eigenvalues. 

If we premultiply Eq. (2.185) by Y’, we obtain 
Y’AX = AYX (2.190) 


If we now take the transpose of Eq. (2.186) and postmultiply the result 
by X, we obtain 


YAK = pY'X (2.191) 
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We now subtract Eq. (2.191) from Eq. (2.190); the result is 

A-pYX = 0 (2.192) 
If the eigenvalues A and p are such that A # », we have the result 

yx = 0 (2.193) 


It is evident from Eq. (2.193) that the eigenvectors of A are orthogonal 
to the eigenvectors of A’ belonging to different eigenvalues. This fact 
may be utilized in the determination of the eigenvectors of nonsym- 
metrical matrices that belong to repeated eigenvalues. 


THE GRAM-SCHMIDT ORTHOGONALIZATION 
PROCESS 


In certain applications such as the determination of the eigenvectors 
belonging to repeated eigenvalues, we are concerned with the problem 
of obtaining an orthogonal set of vectors from any given set of n linearly 
independent vectors. The procedure by means of which this object is 
achieved is known in the mathematical literature as the Gram-Schmidt 
orthogonalization process. 

In this process we are given a linearly independent set of vectors 
X1, Xo, Xg,...,X,. The problem is to find scalars e,; such that the 
vectors given by the scheme 


Y; = xX) 

Yo = €91X, + Xo 

Oe te eee ee os (2.194) 
Y, = Cnr X1 + CygXo + -°- + Cp n-1Xn-1 + Xp 


are orthogonal. 
Let us introduce the notation 


Behe = Sekpe Na: Hj (2.195) 
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so that (Y,,X ? denotes the scalar product of the vectors Y, and X ij Now 
consider the vectors 


¥, = x, 
(Y,, Xo) 
Yo = X2-Y, — 
(¥,,¥) 
(Y,,X5) (Y9,Xq) 
Yg = X3- Yy — Yo — Ce) 
(YY) ana) 
(Yer) 
Veter ke = ey oe 
Pee on Lt) as 
We note that 
(Reexa ikea) 
(Y1,Y 9) = (X,,X_) - ——————_ = 0 (2.197) 
(X,,X)) 
We can show similarly that (Y,, Y,) = 0 and that in general 
(Yj, Y,) = 0 for j#k (2.198) 
Therefore the vectors Y,, Yo, ..-, Y,, form an orthogonal set of vectors. 


Example. Consider the set of vectors 


1 = (1,0, 1,0) 
2 = (1,1,3,0) 
X, = (0,2,0,1) 


The orthogonal set of vectors Y are obtained using Eqs. (2.196) 
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Y, = (1,0,1,0) 


(1, 1, 3,0), (1,0, 1,0) 
(1,0, 1,0), 1,0, 1,0) 


= (1,1,3,0) - 2(1,0,1,0) 
(-1,1,1,0) 


Yo = (1,1,3,0) - (1,0, 1,0) 


(0, 2,0, 1), 1,1, 1,0) ( 

(-1,1,1,0), (-1,1,1,0) 

_ 0, 2,0,0), (1,0, 1,0) 
(1, 0, 1,0), (1,0, 1,0) 


= (0,2,0, 1) - -1,1,1,0) 


wo 
M 


(1,0, 1,0) 


THE MODAL MATRIX AND THE 
SPECTRAL MATRIX 


Let the nth-order matrix A have n distinct eigenvalues 4,, 
k=1,2,...,m and n eigenvectors X,,k = 1,2,3,...,”. Let the n 
eigenvectors X,, be arranged as the columns of a square matrix M. This 
square matrix is called the modal matrix of A; it has the form 

M={[X, X,g Xg... XJ (2.199) 
The modal matrix M is therefore a partitioned matrix whose columns are 
the eigenvectors of the matrix A. 

The spectral matrix of the matrix A is a diagonal matrix S$ of the nth 
order whose elements are the eigenvalues of the matrix A (we are 
considering here the case in which the matrix A has n distinct eigen- 
values). The spectral matrix S has the form 


4 0 0 0 
OeeeAs 20 0 
S=/0 © Ay ... 0 |= diag(,;A,,---,2,) (200) 
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The typical eigenvector and eigenvalue X, and A, of the square matrix 
A satisfies the eigenvector equation 


VS AROS ONS Re a (2.201) 


Let the matrix A be postmultiplied by its modal matrix M to obtain the 
result 


AM = AIX, Xp  Xg---X,] = (AX; AXp AX --- AX, 


" 


2.202 
Doky Shaky Ag Ma eee Ay Sn) mets (2.202) 


The matrix M has an inverse provided that the eigenvectors of the 
matrix A, X,,k = 1, 2, 3,--.,m are linearly independent. This will be 
the case if A has n distinct eigenvalues A, or if A isa symmetrical matrix 
so that A°= A. If the matrix A is not symmetrical and some of its 
eigenvalues are multiple eigenvalues, the matrix M may not have an 
inverse. The result (2.202) AM = MS can be premultiplied by M in 
the case M has an inverse. We thus obtain 


M1 AM = S (2.203) 


Equation (2.203) is said to represent the diagonalization of the matrix 
A to a diagonal matrix. The equation AM = MS can also be postmultiplied 
by the inverse of the modal matrix M7! to obtain 


A = MSM"! (2.204) 


It is interesting to note that if Eq. (2.204) is multiplied by itself, the 
result is 


A2 = (MSM~})(mSM7}) = Ms(M-!M)sm-! = ms?M-1 (2.205) 


The matrix S? is a diagonal matrix whose elements are the squares of 
the eigenvalues of A. If we multiply Eq. (2.204) by itself k times we 
obtain 


At = MS*M-! =k = 0,1,2.3,... (2.206) 


The matrix S* is a diagonal matrix whose elements are the kth powers of 
the eigenvalues of A. 
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THE DIAGONAL FORM OF A SYMMETRICAL 
MATRIX WITH DISTINCT EIGENVALUES 


Let us consider A to be a symmetrical matrix with distinct eigenvalues. 
We now multiply the equation AM = MS on the left by the transpose of 
M, M’, to obtain 


M’AM = M’MS (2.207) 


We now take the transpose of Eq. (2.207), realizing that A’ = A, and 
obtain 


M’AM = SM’M (2.208) 
From Eqs. (2.207) and (2.208) it follows that 

M’MS = SM’M (2.209) 
Since S is a diagonal matrix, in order to satisfy Eq. (2.209) we must have 
M’M = a diagonal matrix. If the eigenvectors X , are normalized to unit 
length so that 

XX Lo for, PFs (2.210) 
the modal matrix of the symmetric matrix A has the property that 


MM = 1 (2.211) 


where / is the nth-order unit matrix. Thus M is orthogonal in this case 
and the Eq. (2.204) takes the form 


A = MSM’ (2.212) 


In Eq. (2.212) M is the modal matrix of A and M’ is its transpose. 
This equation is the diagonal form of a symmetric matrix with distinct 
eigenvalues. 

Diagonalization of a matrix is closely connected with the problem of 
obtaining quadratic forms. In many engineering applications, such as 
finding of the principal axis of inertia and coordinate transformations, it 
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is desirable to reduce a given expression with cross-product terms to a pure 
quadratic form, that is, to an expression involving squares of the variables 
without cross-product terms. It will be shown through the following 
example that this problem is similar to finding the appropriate character- 
istic values and vectors of the expression with cross-product terms as 
described above. 

Example. Obtain the pure quadratic form for the expression 


Q = oe — 2x, Xp + 2x57 — 2xgX_ + oe (2.213) 


The above relation can be put in symmetrical matrix form as follows: 


2x,%_  -% Xp 0 
Q = |-x9x) 2WkxgXQ = X_gXy (2.214) 
0 —XgXg X3Xg 


In the above, it is assumed that x;x; = x;x;. Thus the resulting [A] matrix 
of the coefficients becomes 


2-1 0 
Az=|-1 2 -1 (2.215) 
oo=1_.1 


Q = (x, x x3) J-1 2 -1] | x (2.216) 


Multiplying the last two matrices, we get 


110 Digital Computer Methods in Engineering 


2x1 - Xp 
Q= (xy XQ X3) | -X, + 2xy - xg (2.217) 


—Xo + X3 


which upon multiplication results in 


Q = x1 (2x, - xg) + xg(-xy + By - x5) + X3(-Xy + x3) 
A 2 2 (2.218) 
= 2x,° — 2x, x9 + 2x9” - Box + X3 


Equation (2.218) is identical to the expression for Q given by Eq. 
(2.213). This example was purposely chosen to yield matrix [A] whose 
characteristic values and vectors were previously calculated and are 
given in the normalized form as follows: 


A, = 3.238; (0.591, -0.736, 0.328] first column matrix [M] 


> 
i) 
tl 


= 1.555; [-0.736, —0.328, 0.591] second column matrix [M] 
Az = 0.198; (0.328, 0.591, 0.736) third column matrix [M] 


Using these values, the corresponding [M] matrix of Eq. (2.199) becomes 


0.591 -0.736 0.328 
M =| -0.736 -0.328 0.591 (2.219) 
0.328 0.591 0.736 


And, since [M] is an orthogonal matrix [M’ = M~!] the corresponding 
relation to Eq. (2.203) becomes : 
0.591 -0.736 0.328 2-1 0O 
-0.736 -0.328 0.591 -1 2 -1 


0.328 0.591 0.736 0-1 #1 


(2.220) 
0.591 -0.736 0.328 3.238 (0 0 


-0.736 -0.328 0.591] =| 0 1.555 0 
0.328 0.591 0.736 0 0 0.198 


— 
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Consider the following transformation from x to € coordinate system 


a 0.591 -0.736 0.328] | é, 
x_| = |-0.736 -0.328 0.591] | é (2.221) 
X3 0.328 0.591 0.736 é3 
In matrix notation this becomes 
(x) = (M]( (2.222) 
Transposing this, we get 
(x)’ = [&\(M)’ (reversed order) (2.223) 
We now write Eq. (2.223) in full matrix form: 
0.591 -0.736 0.328 
(4, Xp, Xq) = (Ey 9&5) |-0-736 -0.828 0.591 (2.224) 
0.328 0.591 0.736 


Using the expressions given for matrices X 


and (2.224) in Eq. (2.216), we get: 


0.591 


Q = (Ey, &y &3) |-0.736 


0.328 
0.591 
—0.736 
0.328 


—0.736 
—0.328 
0.591 


—0.736 
—0.328 
0.591 


and X° in Eqs. (2.221) 


0.328) | 2 -1 0 
0.591] |-1 2 -1 
0.736} | 0 -1 0 

2.225 
0.328 | | é, : , 
0.591 | | &5 


0.736 
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Substituting the diagonal matrix of Eq. (2.220) in place of the three 
matrices at the center of the above expression, we get 


Q = 3.238€,? + 1.555€,? + 0.198€,” 


which represents the quadratic form of Eq. (2.213) with the transfor- 
mations of coordinates x to & as given by Eq. (2.221). The transfor- 
mation from the coordinate system € to x can easily be obtained by 
premultiplying Eq. (2.222) by [M]~!(= [M)’); this gives 


é = Mx (2.226) 


PROBLEMS 


1. Find the eigenvalues and eigenvectors of the matrices 


a eas | -2 -1 0 i 30'= “6 
As/[s @ 2 B=|{1 2 8 € =/1/0- 2 
0-1 5 4 5 6 D. 7 


2. Given the matrix 


2-2 3 
show that its eigenvalues are 
A =]1 1 1 N 1 
y= Lg =-2,, = 3 
Les. 1 


3. Given the matrix 


show that A, = 1, Ay = 1, Ag = 2 


A= 


4. Given the matrix 


Aug show that A, = 3, Ap = 3, and Ag = 12 


10. 


1b 


12. 


13. 


14. 


15. 


16. 


Eigenvalues, Eigenvectors, and Quadratic Forms 113 


Find the characteristic polynomial of Prob. 2 by the method of 
A.N. Krylov. 

Find the characteristic polynomial of Prob. 3 by the method of 
A.N. Krylov. 

Find the characteristic polynomial of Prob. 4 by the method of 
A.N. Krylov. 

Find the highest characteristic value and the corresponding vector 
for the matrix 


1 1 1 

tag o3 Answer: characteristic value 5.042 vector (5 
iterations) 0.555, 1.246, 1,000 

Nike. 42 


by matrix iteration. 

Find the smallest characteristic value of the above problem by 
iteration. 

Using orthogonality relations find all of the characteristic values 
and vectors of Prob. 2. 

Obtain the characteristic values and vectors of the matrix 


1-4 2 
-4 1 -2 
2-2 2 


by iteration. 

Obtain the characteristic equation and characteristic vectors of 
Prob. 2 by the method of A. M. Danilevsky. 

Obtain the characteristic equation and characteristic vectors of 
Prob. 3 by the method of A. M. Danilevsky. 

Obtain the characteristic polynomial and characteristic vectors of 
Prob. 4 by the method of A. M. Danilevsky. 

Obtain the characteristic equation of Prob. 8 by the method of 
A. M. Danilevsky. 

Solve the cubic equation obtained above and use the method of 
Danilevsky to obtain the characteristic vectors. 
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OT; 


18. 


19. 


20. 


21. 
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Given the matrix 


"4 cos (9) sin (6) 
~ | -sin (6) cos (0) 


show that A is orthogonal, A’A = I, and determine the inverse of A 
and its eigenvalues and eigenvectors. 
Given the three linearly independent vectors, 


by the use of the Gram-Schmidt process construct a set of three 
orthogonal vectors Y,,, k = 1, 2, 3 from X,, Xy, X3. 
Reduce the matrix 


78 -60 15 
A=|150 -117 30 
200 -160 43 


to the diagonal from M-! AM = S. 
Given the matrix 


5 -1 -1 
feo) (ie eae fas | 
-2 2 4 


show that the eigenvalues are 2, 4, 6. Determine the eigenvectors 
of A. 
Show that the eigenvalues of 


22. 


23. 
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a a2 0 
ie r(r + 1) 0 ae 
2 
0 r(r + 1) P 
2 


are a, (1 + r)a, -ra. Find the corresponding eigenvectors. 
Show that the matrix 


LD Oe Oi t 

et “0 «9 © 
A= 

0-1 0 0 

0 0-1 O 


is orthogonal. Find its eigenvalues and verify that its eigenvectors 
are “vectors of zero length.” 
Obtain the quadratic form of the equation 


Q = 2x,2 + Qxy? — 15x47 + 8x, xy - 12xgxq - 12x, x5 


and the corresponding transformations. 
Hint: The characteristic values of the resulting matrix are —2,—-18, 
and 9. 
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3 ROOTS OF 
POLYNOMIAL AND 
ALGEBRAIC EQUATIONS, 
LAGRANGE’S 
INTERPOLATION 
FORMULA, METHOD OF 
LEAST SQUARES 


The formulation of many engineering problems results in linear con- 
stant coefficient differential equations, the solutions of which can be 
obtained by writing the characteristic equation of the differential equation 
and solving for the roots of the resulting polynomial equation. In 
addition, the natural frequency of vibration in many electrical and 
mechanical systems is obtained by solving the characteristic equation of 
the system, which usually is an algebraic polynomial. In control systems, 
the stability of the system is determined by location of roots of the 
denominator of the transfer function of the system. In all of these 
cases, one is faced with solving for the roots of a polynomial equation. 
In a number of applications, solutions will result only by finding the 
roots of algebraic equations which, in addition to powers of the inde- 
pendent variable x, may include trigonometric and/or exponential 
functions (transcendental equations). 
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The inverse of finding the roots of polynomial equations, namely, given 
a number of data points, find a polynomial which passes through those 
data points, also occurs. In this respect, Lagrange’s interpolation formula 
is the one most often used. However, in problems where the data may 
include random measurement errors, the method of least squares is often 
employed. 


ROOTS OF POLYNOMIAL EQUATIONS 


Consider the nth-order polynomial equation P,, (x) of the independent 
variable x 


f(x) = PQ) = Ajx™ + A,_yxt-1 +--+ Aya + Ay = 0 
(3.1) 


where the A’s are the coefficients of powers of the independent variable 
x. 

Several numerical methods are available for determining the roots of 
this polynomial equation. However, before describing these methods a 
number of algebraic theorems dealing with the roots of polynomials are 
given for reference. The proofs of these theorems can be obtained in 
standard texts on algebra. 

Theorem 3-1. If (x — r) is a factor of f(x), then r is a root of the 
equation f(x) = 0. 

Theorem 3-2. If the polynomial f(x) is divided by (x - r), the re- 
mainder will be f(r). For the proof, let f(x) be divided by (x - r) and 
call the resulting polynomial Q(x) and the remainder R. This gives 


f(x) = (x - NQ(x) +R (3.2) 
forx =r 
{7 =R 
Theorem 3-3. If r is a root of the equation f(x) = 0, then (x - r) isa 
factor of the polynomial f(x). 


Theorem 3-4. A polynomial of degree n cannot have more than n 
roots. 
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Theorem 3-5. If f(x) is a polynomial and a and b are real numbers such 
that f(a) > Oand f(b) < 0, then f(x) has a root between a and b. 

Theorem 3-6. The coefficients of the polynomial are related to the n 
Toots 71, Tp, Tg, +++ Ty bY 


An-1 
= 1, = -(ry + Tg +13 +--+) 
A, i=l 
A,-2 
A = igre Mage Se yg hy Sate Rg As ss. 
i<j 
2 (3.3) 
A,-3 y 
miss Tit jTe = —MyTelg - Milatl4 - °° 
A, isjck 
A n 
ee <1? y TT Tees = (-D"ryrgrgty ee Yb II ;, 
A pa ! =1 
i fejekesss 


Theorem 3-7. If the coefficients A, of P,,(x)are real anda + bj (where 
j? = -1) isa root of polynomial P,, (x) = 0, thena — bj is also a root. 

Theorem 3-8 (Rule of Signs). The number of positive roots of 
P(x) = 0, where the coefficients are real, cannot exceed the variation in 
sign of the coefficients of the polynomial P,,(x) = 0. The number 
of negative roots cannot exceed the number of variations in sign of the 
coefficients of P,,(—x). Moreover, in each case, if the number of roots is 
less than the number of variations in sign, it must be smaller by some 
multiple of two. 

Theorem 3-9. To transform an equation of the nth degree into an 
equation each of whose roots is the negative of the corresponding root 
of the original equation, we replace the independent variable x by -x in 
the equation. 

Theorem 3-10. To transform an equation of degree n into an equation, 
each of whose roots is less by & than the roots of the original equation, 
we proceed as follows: Divide the polynomial by (x — k) and denote 
the remainder by Rp; divide the quotient by (x — k), and denote the new 
remainder by R,; continue until n+ 1 remainders are found. Then 
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Rey Rye gy ee Ry + Ry = 0 


n- 


is the required equation. This can be accomplished through synthetic 
division as described below. 


SYNTHETIC DIVISION 


Consider the polynomial 


f(x) = Pix) = Ayx™ + A,_yx™ 71+... + Ayx+ Ay = 0 
(3.4) 


The following relations can be written using the coefficients of this 
polynomial 


P(x) = A, 


P, (x) = xP (x) + Aye 


Po(x) = xPy(x) + A, (3i5)' 
P(x) = xP, _ (x) + Ay 
Multiplying the first, second, third, . . . equations above by 
x”, x"-1)yn-2 | respectively, and adding all of the resulting relations 


will give Eq. (3.4). In this operation, the terms cross referenced by 
arrows in Eq. (3.5) will drop out. Thus the value of polynomial P_, (x) at 
specific value of x = x, can be obtained by evaluating Po (xq) through 
PR, (x) in the above relations. This can be accomplished by synthetic 
division and is considered simpler than substituting x, in Eq. (3.4) and 
evaluating powers of x). The synthetic division will result in 


A A 


n n=l An-2 0°" Ay Anos / 40 


XqPo(x%p) Xp Py (Xp) --- XQ P,_9(%) Xp Py -1 Xp) 


Po (xq) P, (x9) Po(xy) «+. P,_1(%) P,, (x9) 
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where values of P, through P,, are sums of respective columns, for 
example, P, = x)P, + A,_» and values at the second row are obtained 
by multiplying P), P;, Po, --- by Xp. 

Example. Determine the value of f(3) for the function 


fx) = 45 — 30x3 + 6x? - 9 
by synthetic division. This results in 
4 0 -30 6 0 -9 [3 


12 386 #18 72 216 
4 12 6 24 72 207 = f(3) 


DERIVATIVE OF POLYNOMIALS 
BY SYNTHETIC DIVISION 


Differentiating Eqs. (3.5) with respect to x, we have 


Para 0 
Pi (x) = Po(x) + xP4 (x) = Po(x) = A, 
P5(x) = xP} (x) + Py (x) (3.6) 


Pe (ay = BES eax) HBO tx) 


where primes denote derivatives with respect to x. 
Further differentiation of the above relations will result in 


Pe = 0 


Pi(x) = 0 


n 


P3(x) = xP{(x) + 2P}{(x) = 2A 


POG) = xP" _ 4 (x) + 2P7 


n- 1%) 
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or 


Pox) [PT] 
55 6 becca bt Pi_ (x) 
2 P} 


wherei=1, 2, 3,... 
Further differentiation of the above relations will result in the values 
of third- and higher-order derivatives 


Baya eee) oe 
3! 4! 


These derivatives can be obtained by extending the synthetic division 
previously described: 


n n-1 
XoPo — XoP1 --- XoPn-4 = X0Pn-3  X0Pn-2 © *0Pn-1 
Lean eig Py P,-3 Ph-2 Py-1 P,, (xq) 
XP, XqgP2 +++ XPh-g  XpPn-2  X0Pa-1 
Ey Py Ps Ph-2 Pa-1 P,, (x9) 
S Po P3 ¥ en Peed 
Malte I ay ov2! 
PY PS Py Peay P* (x9) 
2! oat 2! 2! 2! 
oe Bi 
Bare oiler ary 
PS Py PS P(x) 
3! 8! 3! 3! 


These results, together with Taylor’s series expansion of P(x) around h 
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2. 3 
P(x +h) = P(h) + xP(h) + Fried + aro Ss (3.7) 


where h is an incremental quantity, can be used to obtain polynomials 
whose roots differ from the roots of a given polynomial by the constant 
factor h. This in turn forms the basis of Horner’s method of finding 
real roots of polynomial expressions to be described shortly. 

Example. Given the example 


P(x) = x3 — 12x? + 47x - 60 = 0 


write an equation with roots diminished by 2. If the roots of the above 
equation are r,, ry, ..., the roots of 


P(x +h) = x3 + Blh)x? + C(h)x + D(h) = 0 


where B(h), C(h), and D(h) are functions of h, will be (r, — h), (rg — h), --- 
since P(r; - h + h) = P(r) = 0. Thus, we need only to obtain the 
coefficients of the Taylor’s series (3.7) by synthetic division. This 
gives 


2-20 54 
1-10 27 -6 = Pi) 
2 -16 
£8) = PA) 
2 
1 -6 = P%(2)/2! 
1 = P%(2)/3! 


Having this, the desired equation becomes 


P(x + 2) = x3 - 6x? + lx -6 = 0 
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HORNER’S METHOD OF FINDING REAL 
ROOTS OF POLYNOMIALS 


This method involves finding the approximate location of the real 
root of a polynomial and successively obtaining polynomials whose 
roots are diminished by these approximate amounts. This method was 
primarily devised for paper and pencil calculations. For calculators and 
computers it is much longer than other methods that will be discussed 
in this chapter. The method is best illustrated by an example. It is 
desired to find the roots of the equation 


f(x) = x3 + 9x2 — 20x - 90 = 0 (3.8) 


To obtain the approximate location of a root, set x = 0,1,2,... . This 
will result in ((3) = —42 < 0 and (4) = 38 > 0. Thus, by Theorem 3-5, 
f(x) has a root somewhere in the region 3< x < 4. The next step is to 
find a polynomial whose roots are diminished by 3. That is, 


1 9 -20 -90 /3 


3 6 48 

1 2 16 42 
3 45 

1 6 61 
3 

1 18 


This results in the polynomial 
f(x) = x3 + 18x? + 61x - 42 = 0 (3.9) 


The roots of this equation are between 0 and 1. Neglecting higher-order 
terms, let us take x = 42/61 ~ 0.7 and evaluate f(0.7) by synthetic 
division: 


UF 61 -42 [9.7 


0.7 13.09 51.863 
1 18.7 74.09 9.863 = f(0.7) 
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From this f(0.7) = 9.863 > 0, let us try x = 0.6; this results in f(0.6) = 
1.296, again greater than zero. Let us now try x = 0.5; this will give 
f(0.5) = —6.875, which is negative. Thus, the root of Eq. (3.9) is 
between 0.5 and 0.6. Again by synthetic division we write an equation 
with roots reduced by 0.5 from those of Eq. (3.9). This results in 


1. 18 61 -42 [9.5 


0.5 9.25 35.125 


1 18.5 70.25 -6.875 
0.5 9.5 

LT 19.0 °79.75 
0.5 

1 19.5 


with the equation 


f(x) = x3 + 19.5x? + 79.75x - 6.875 = 0 


The roots of this equation are between 0 and 0.1. Repeating the above 
process will give the final value of the root of Eq. (3.8) to three signifi- 
cant figures as 3.584. 


NEWTON'S METHOD OF SOLVING POLYNOMIAL 
AND TRANSCENDENTAL EQUATIONS 


This method of calculating the roots of polynomial and transcendental 
equations is used in both desk-type calculators and pencil and paper 
calculations. It involves successive approximations to the roots. New- 
ton’s method can also be extended to determine the complex roots of a 
polynomial equation. However, in this case the approximations should 
start with complex values. In this respect the method suffers from very 
complicated numerical work in obtaining the complex roots of a poly- 
nomial equation. The following is a description of the method for 
obtaining the real roots of transcendental and polynomial equations. 

Suppose that A is a real root of the equation f(x) = 0, and that A, is an 
approximation to A. Also, let us define the difference by h 
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h=A-A, 
(3.10) 
A = A, +h 
Taylor’s series expansion for f(x) results in 
; 1 
f(A) = f(A, + h) = f(A) + Af (Ay) + rie oo (3.11) 


But since A is a root of f(x), then f(A) = 0; neglecting terms containing 
2 and higher, we have 


(Ay) + nf (A,) = f(A,) + (A - Aj) f(A,) = 0 


From this 


which is the next approximation to A. Continuing, we arrive at a value 
for the root. 

Example. Find the root of the equation f(x) = 2 cos x — 3x = 0. For 
x = 0, f(0) > 0; and for x = 7/2, {(7/2) < 0. For the first approximation, 
take midway between 7/2 and zero, that is, A; = 7/4 ~ 0.70. This gives 


f(Ay) 
Aen Ise 0.7 - 2 cos (0.7) - 2.10 


i iene telites = 0.567 
f(A) -2sin0.7 - 3 


The next approximation gives 


Further approximations can be obtained by repeating this process. 
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GRAFFE’S METHOD OF OBTAINING 
ROOTS OF POLYNOMIAL 
EQUATIONS 


This method is based on the fact that if the roots of a polynomial 
equation are different from each other, this difference will be exaggerated 
if the roots are squared. A polynomial each of whose roots is the square 
of the roots of the original polynomial can be written. Thus, by re- 
peating this process the roots of the resulting polynomials will be 
widely separated or, mathematically, r,; > ry > rg --- where ry, rg, rg, - + 
are the roots of the succeeding polynomials. As we will see, this makes 
it possible to use approximations to Eq. (3.3) as a basis of finding the 
roots. The Graffe method has the advantage of yielding the real as well 
as the complex roots of a polynomial. In addition, it results in all of the 
roots rather than one root at a time as in the case of previously given 
methods. As mentioned before the method is directly applicable when 
the roots are sufficiently different from each other. If the roots are 
close to each other in magnitude (or complex conjugate), special pro- 
cedures should be used to obtain their values. These procedures will be 
outlined later. 


Assuming the variation r; > ry > r, > r, --- in the roots of the 
1 2 3 4 
polynomial 
fx) = A.x? +A, xt} 4 A, _oxt-2 4 +--+ A, 4x + Ay = 0 
(3.12) 


Taking the dominant parts of Eq. (3.3), we can relate the above coeffi- 
cients and the roots 


A 


n=1 


or A,-2 A,-3 
A esawe Z A 


n n n 


vee (3.13) 


ee 


These relations are obtained from Eq. (3.3) by neglecting smaller-order 
terms. As mentioned before, the roots of any polynomial equation can 
be separated by squaring. Take as a preliminary example the quadratic 
equation 


meh aioe 82) yO (3.14) 
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with roots r, = 1, r, = 2. Squaring this equation will result in 


(x? + 2)? = (3x)? 
x* — x2 4 4 = 0 


Substituting y = x2 in this equation we obtain 


i 


y? - 5y + 4=0 


with roots 1 and 4. Repeated squaring of the equation will give roots of 
1 and 16, 1 and 256, and so on. The corresponding values of the roots 
of the original equation can be found by using relations (3.13). This 
results in 


2 82+2=0 ri) = 3 ro!) = 0.667 


tad 
! 


o 


y? - By + 4 7'2) = V5 = 2.296 r9{2) = 0.894 


nN 
I 


2_ 17z+16 = 0 7103) = VIT = 2.081 193) = gf 


= 0.985 


2 _ 257w + 256 - 0 1,4) = Y257 = 2.001 7,4) = v= 


= 0.995 
(3.15) 


Thus, the resulting roots obtained by computing the coefficients are close 
to the roots of the original Eq. (3.14). 

The coefficients of Eq. (3.15) can be evaluated systematically as 
follows. Consider the polynomial equation 


&§ 
t 


f(x) = x™ 4 bes + Banta + bax" 8 fee + b,x + b, = 0 
(Note that in this equation the order of subscripts of b’s is reversed as 
compared to Eq. (3.1). For example, the coefficient of the x” ~ 1 term in 
Eq. (3.1) was A, _,, while in the above equation it is b,. This is done to 
ease the explanation which follows.) Transposing odd and even terms in 
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x to the right and left sides of the equal sign, we get 


(x” + [goo + re aie + --) 


= -()x"") + bax > + bexeae + ---) 


Squaring both sides of this equation, we obtain 


x2n _ EECA (ne = 2by) + xan (6? = 2b, bg + 2b) (3.16) 


2n-6 (p 2 
- x2n (bs = 2bg by + 2b, bs - 2bg) + Email 
Substituting y = -x? in the above results in 
y+ y“1(by2 ~ 2by) + y"-2 (bg? - 2b, bg + 2b) +--+ = 0 


These coefficients can be obtained from the tabulated form 


First row ell by by bg bg 

Second row 1 6? 6,2 b,” bye 
Third row -2b, -2b, bs ~2by by -2bg bs 
Fourth row 2b, 2b, bs 2bg bg 
: -2be -2b, b, 

2bg 

y= -x2 1 by2=2b, b,2-2b,b, b,2-2bob, 4? - 2bg bs 

+2b, +2by bs + by be 
~2be ~ 2b; by 

2b, 


The coefficients of x in the original polynomial are written on the 
first row. The squares of these are written on the second row. The 
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elements of the third row are obtained by multiplying the elements of 
the first row on either side and multiplying the result by —2. The ele- 
ments of the fourth row are obtained by multiplying the second element 
of the first row on either side and multiplying the result by 2. Similarly, 
the elements of the fifth, sixth, and other rows are obtained. The 
coefficients of the equation whose roots are the square of the roots of 
the original equation are obtained by summing the columns of the table 
from second row and below. Using this method for finding the coeffi- 
cient, the root-squaring method of Graffe can be applied in finding real 
and complex roots of polynomial equations, as illustrated in the following 
examples. 
Example. Find the real root of the equation 


x3 — 35x? - 206x + 240 = 0 (3.17) 


The table of coefficients for this problem can be written 


5.76 x 104 
0 


1.225 x 10° 
0.412 x 10° 


r 1.637 x 10° 5.92 x 104 5.76 x 104 
2.68 x 10° 3.50 x 10? 3.32 x 10? 
0.12 x 108 0.19 x 10? 0 
2.56 x10° | 3.31 x10? 3.32_x 10? 
6.55 x 10!? 1.096 x 10!9 | 1.102 10! 
0.006 x 10!? 0.0008 x 10!9 


6.54 x 10!2 1.095 x 10!9 | 1.102 x 1019 


From the last row we have 


Ir)| = V6.54 x 1012 = 39.99 ~ 40 


19 

\ro| = ,/ 1.095 x 10°" = 5.997 ~6 
V 6.54 x 1012 
19 

(relapse US 0000; 
1.095 x 1019 
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The root-squaring method is terminated when the resulting roots are 
sufficiently separated, that is, when the original condition of the method 
is fulfilled. Another way of stating this is whether additional squarings 
will separate the roots further. As seen from the above table the amount 
of improvement in the value of the roots is indicated by the cross- 
product terms, that is, -0.006 x 10!? compared to 6.55 x 10!? and 
-0.0008 x 10!% compared to 1.096 x 1019 in the last iteration. The 
squarings are terminated when the contributions of cross-product terms 
are considered negligible as compared to those of regular terms. 

The above absolute values can also be computed using logarithms. It 
can be seen by substitution in Eq. (3.17) that r; = 1.0009 is a root. 
From Eq. (3.3) we have 


Ty + Te tls = 35 
TyTg + TyIg + Tels = — 206 
TyTgrg = —240 


Since rg is +1 from the last equation above, ry and ry must have different 
signs; and from the first equation r; should be +40 and ry should be —6. 


Complex Conjugate Roots 


For polynomials with real coefficients the complex roots occur in 
conjugate pairs. Since the magnitude of a pair of complex conjugate 
roots is the same, no amount of root squaring will separate these roots. 
In these cases, the real roots of the polynomial are calculated first and 
the complex conjugate pair of roots is determined by using the real 
roots and the products of complex conjugate roots. Assume that the 
following polynomial is obtained after several root squarings 


Bo sB. yet Be oat? fos By = 0 (3.18) 


Using Eqs. (3.13) the roots of the polynomial can be written 


mee B,-1 
rape a 
n 
Soda B,-2. oe B,-2 
172 2 ia 
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rire B,-3 = B,-3 
ora = ‘3 Ki 
B, B,-2 
Tan Tak PEs r pie! 
2 Sa = ? , 
B, Bn 2 (3.19) 


Assuming r, and r, to be real and ry and rz to be complex-conjugate 
pairs, the second of the above relations will involve imaginary numbers. 
In the root-squaring process this fact results in nonvanishing of cross 
products in the corresponding B,, _ 5 term, as will be seen in the following 
example. In the Graffe method the presence of a complex-conjugate 
pair of roots is indicated by the nonvanishing of cross-product terms 
and many times also by frequent change of sign of these terms. 
Example with complex roots. Find the roots of the equation 


x3 + 94x? — 575x + 2,500 = 0 (3.20) 


The table of coefficients for this equation becomes 


8.836 x 10° 3.30 x 10° 


1.15 x 10° — 4.700 x 10° 

a 9,986 x 10° = 1,394 x 105 6.25 x 10 
9.972 x 107 1.943 x 10! | 3,906 x 1013 
0.028 x 107 -12.483 x 1010 


10.0 x 107 =10.540x 10!9 | 3.906 x 1013 


From the last row 
Iry| = ¥108 = 100 


Since this is the absolute value of the root, by substitution in Eq. (3.20) 
we note that r, should be negative r, = -100. The nonvanishing of the 
cross-product term in B, _ 9 (-12.483 x 10!°) indicates the presence of a 
complex root and since (ryry)* = —10.54 x 10°, then T. must be 
imaginary. By Theorem 3-7 rz must also be imaginary. Assuming that 
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Tg = ut jv 


: A ato (3.21) 
oF oe 
we obtain 
2. .2 _ ‘tats 3.906 x 1019 
ToTg = Ue + Uo = = Wo _ = 85 
r 8 
1 10 
and since 
Ty + fo +o = —94 1a + T. = 6 
Dee es aes: 


This together with Eq. (3.21) will result in 2u = 6 or u = 3, and from 
u? + v2 =25,v=4. Finally, the roots of the equation can be written 


r, = -100 1,73 = 38 + 4j 


The presence of real roots of equal magnitude will similarly be indi- 
cated by nonvanishing of the cross-product terms. In general, the root- 
squaring method is unsatisfactory because of special rules it requires in 
case of multiple roots and complex roots. In the case of all real roots 
the application of the method is straightforward although the signs of 
the roots should be determined separately by direct substitution in the 
equation. In a given problem the real roots can be obtained by, for 
example, Newton’s method. Each root can be factored out of the 
equation by synthetic division and the process repeated on the poly- 
nomial of lower degree. If the polynomial contains only a pair of 
complex-conjugate roots, the quadratic obtained after extracting all of 
the real roots can be solved for complex roots. Another often-used 
method of obtaining the roots of polynomials by digital computers 
involves successive extraction of quadratic factors by iteration and the 
solution of the resulting quadratic equations for the roots which may 
be complex conjugate. The quadratic polynomials can be found by 
Lin-Bairstow’s method, described below. 
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LIN-BAIRSTOW’S METHOD OF 
QUADRATIC FACTORS 


Consider the polynomial 


Pix) = A,x" + 7S Wr ee + Ay ox * $oaee 


(3.22) 
+ Agx? + Ayx + Ay = 0 


It is desired to extract the quadratic factor x? + r*x + s* from this 
polynomial. If P_, (x) is divided by the trial polynomial x? + rx + s we 
obtain a quotient polynomial Q(x) of degree n—2 and aremainder Rx + S 


P(x) = (x2 4 rx + 9)(B,x"~? + Beene + Beasts fee 


+ Bax + By) +Rx+S @.23) 


Equating the coefficients of the like powers of x in Eqs. (3.22) and 
(3.23), we get 


Heil. 

not = B24 +B, 

n-2 = B,9 + 7B,_, + 8B, (3.24) 
A, = R+ rBy + sBg 


Ay = S4+ sBy 
The coefficients B of the factored polynomial and the terms R and S 


are functions of r and s. Assuming that x? + r*x + s* is a factor of 
the polynomial P_ (x), then 


Rir*,s*) = 0 S(r*,s*) = 0 (3.25) 


To find r* and s* we take values of r and s near these values, so that 


re = r+ Ar 


i 


(3.26) 


s* = s+As 
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Using Taylor series expansion we have 


R(r*, s*) ~ R(r,s) + Ar OR “ As FR = 0 
or ds 
(3.27) 
Str*, of) x Str,0) + Ar SS 4 Ae - 0 
or ds 


where the derivatives are evaluated at r and s. Equation (3.27) gives two 
equations in two unknowns Ar and As. Use of these values in Eq. (3.26) 
will give the next approximation to r* and s* and the process is repeated 
from this point. 

The above procedure can be mechanized for a digital computer solu- 
tion as follows. From Eq. (3.22), if we use the last three terms as the 
first approximation of the quadratic form, we get the value of r and s 


i 
eral 
wo ie 


(3.28) 


to es 


N 


Equation (3.22) can also be divided by x” -2 and the resulting second- 
order equation used for the initial approximation of the quadratic 
form. This results in the values of r and s 


A, (3.29) 


Having the values of r and s, solve Eqs. (3.24) for B’s 
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B, = A, 
Foxit Vela Shy aa tey 
n~g = A,.9 ~ 1B, 4 — 8B, 
: (3.30) 
Bop 2 Apc; = 73, pea = SB, _i+9 
By = As - Bg - sB, 
2 = A, - rB3 - sBy 
From the last of Eqs. (3.24) compute 
R = A, ='rB, = sB. 
1 
i ssimcis (3.31) 
S = Ay - 8B 


Noting that B’s are functions of r and s, the partial derivatives of R and 
S with respect to these variables can be computed as follows: 


OR OB 6B, 
— = -(By + r—] - s— 


or or or 
oB OB 
eee 
: 2 as (3.32) 
aS OB, 
saree 
or Or 


OB 
Gah -(6 . —) 
ds ds 


The above equations involve the partial derivatives of B’s with respect 
to r and s. Denoting these partial derivatives by C’s, we have, from 
Eqs. (3.30)* 


*Note that 4’s are not functions of rand s. 
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W 


(3.33) 


" 


bea OB; OB... 
Ghee n-t Bet n ew n-it+2 
or or or 


= -(B, gy t Oy nted —8Cnote2 


Cy = -(Bg + rC3) - sC4 


Partial derivatives of B’s with respect to s, D’s, can also be obtained ina 
similar fashion 


dB 
D,=—=0 
ds 
0B OB 
D te n-1 = Spee -0 
ee ds ds 
OB, - i By i+ OB, -i+2\ (3.34) 
D,-i = S(t (Fe 
ds ds ds 


SD pel) (Bp aamet 8p ayo) 


OBy OB, OB, 
Dy = =-r -(B,+s 7 = -rD, - (By + sD4) 


Substituting these in Eqs. (3.27) and using Eq. (3.32), we get 


Ar [-(By + ry) - sC3] + As [(B3 + sDg) - 1D» | = -R 


(3.35) 
Ar [-sCq | + As -(By + sD,)| = -§ 
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Denoting the coefficients of Ar and As by T, U, V, and W in the above 
equations, we get 


Ar(T) + As(U) = -R 
(3.36) 
Ar(V) + As(W) = -S 
Solving these simultaneous equations for Ar and As, we have 
Meee -RW + SU 
TW - UV 
(3.37) 
Ape -TS + VR 
TW - UV 


These corrections are applied to the assumed values of r and s and the 
process is repeated from relations (3.30). The iterations are stopped when 
sufficient accuracy is achieved between successive values of r and/or s. 
Having the values of r* and s*, the roots are calculated using the 
relation 


—r* + (Pr) (r*) — 45* (3.38) 
Roots = Se an 


Since with these values of r* and s* the remainder terms R and S of 
Eq. (3.23) are zero, the values of B’s will represent the coefficients of 
a (n — 2)-degree polynomial. The roots of this polynomial can be 
calculated by repeating the above procedure. 


TYPICAL COMPUTER LOGIC FOR THE 
MECHANIZATION OF LIN-BAIRSTOW 
METHOD 


The above equations can be written in flow chart from and subsequently 
programmed for digital computer solution. The following flow chart 
describes the logical evaluation of the given equations. The computer 
program is written in BASIC [4] programming language for a GE 235 
time-sharing computer. 

The following computer program mechanizes the logic given in the 
flow chart. The statement numbers corresponding to the blocks of the 
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Flow Chart of Lin-Bairstow Method 


Enter the degree of polynomial and 
the coefficient into the computer 


Compute initial values of 
rand s (Eq. (3.28)) 
ComputeB, C,and D 
(Eqs. (3.30), (3.33), (3.34)) 
Compute R, S, Ar, As 

(Eqs. (3.31), (3.37)) 


Correct values of r and s 
LCT ow =lola + Ar 


Siew = Sold + 8 


Is Ar sufficiently small? 


Yes 
Compute roots 
(Eq. (3.38)) 


Put the polynomial with B 
coefficients of Eq. (3.23) in 
the form of P,, (x)of Eq. (3.22) 


New polynomial 
degree 2 or less? 
Yes 


Solve the second- or first- 
degree polynomial 


Statements 
110 through 170 


Statements 
200 and 210 


Statements 
220 through 320 


Statements 
330 through 400 


Statements 
410 and 420 


Statement 430 


Statements 
450 through 490 


Statements 
500 through 550 


Statements 
560 through 570 


Statements 
580 through 610 
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flow chart are given next to each block. The roots of the following 
polynomial were calculated with this digital computer program 


x® 4 3x4 — 4x3 _ 98x? 4 48x + 65 = 0 (3.39) 
The roots as given at the end of the program are 
Dicky wile 3:25 


and —1, where j? = -1. Some of the notations of the computer program 
are different from those used in the equations. These are 


Equation notation r Ss oh S Ar As 
Computer notation R S Ri Sl R2~ S82 


From the above discussion it can be concluded that the Lin-Bairstow 
method of computing the roots of a polynomial equation is adaptable 
to digital computer programming logic. It is also seen that this method 
gives complex roots of polynomials without having to deal with complex 
numbers during the course of calculation. 


COMPUTER PROGRAM FOR LIN-BAIRSTOW’S 
METHOD 


100 DIM A(10),B(10),C(10),D(10) 
110 READ N 

120 PRINT “DEGREE” N 

130 PRINT “COEFFICIENT” 
140 FORI=NTOOSTEP—1 
150 READ A(I) 

160 PRINT A(I); 

170 NEXT I 

180 PRINT 

190 PRINT 

200 LET R=A(1)/A(2) 

210 LET S=A(0)/A(2) 

220 LET B(N)=A(N) 

230 LET C(N)=0 

240 LET D(N)=0 

250 LET B(N-1)=A(N-1)-R*B(N) 


260 
270 
280 
290 
300 
310 
320 
330 
340 
350 
360 
370 
380 
390 
400 
410 
420 
430 
440 
450 
460 
470 
480 
490 
500 
510 
520 
530 
540 
550 
560 
570 
580 
590 
600 
610 
620 
630 
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LET C(N-1)=—B(N) 

LET D(N-1)= 0 

FOR I=2 TO N—2 

LET B(N-I)=A(N-1)-R*B(N-1+1)-S*B(N-I+2) 
LET C(N-I)= -B(N-I+1)-R*C(N-I+1)-S*C(N-1+2) 
LET D(N-I)= -B(N-I+2)-S*D(N-1+2)-R *D(N-1+1) 
NEXT I 

LET R1=A(1)-R*B(2)-S*B(3) 

LET S1=A(0)-S*B(2) 

LET T=-B(2)-R*C(2)-S*C(3) 

LET U= -B(3)-S"D(3)-R*O(2) 

LET V=-S*C(2) 

LET W= -B(2)-S*D(2) 

LET R2 = (-R1*W+S1*U)/(T*W-U*V) 

LET S2= (-T*S1+V*R1)/(T*W-U"V) 

LET S=S+S2 

LET R=R+R2 

IF ABS (R2) <.000001 THEN 450 

GO TO 220 

LET G=R*R-4*S 

IF G<O THEN 490 

PRINT “ROOTS"-R/2”+OR-"SQR(G)/2 
GO TO 500 

PRINT “ROOTS"-R/2” +OR-"SOR(-G)/2"J” 
LET N=N-2 

PRINT 

IF N=0 THEN 630 

FOR I=N TO OSTEP—1 

LET A(I)=B(1+2) 

NEXT | 

IF N>2 THEN 200 

IF N<2 THEN 610 

LET R=A (N-1)/A(N) 

LET S=A (N-2)/A(N) 

GO TO 450 

PRINT “ROOT”-A (N-1)/A(N) 

DATA 5,1,3,-4,-28,43,65 

END 


DEGREE 5 
COEFFICIENT 


1 


3 4 -28 43 65 


ROOTS 2 +OR-1. J 
ROOTS-3 +0OR - 2. J 
ROOT —1. 
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LAGRANGE'S INTERPOLATION FORMULA 
AND INVERSE INTERPOLATION 


Assume that Yor Vyr Yar +++ ¥, are values of y = P(x), an unknown 
polynomial, at x9, x4, X9,..-,x, where the intervals between the x’s 
(x; — x;) are not necessarily equal. The problem is to find the poly- 
nomial P(x) which passes through (x9, Yq), (x1, 94), (%¥g) Yq), +++ (te ad 
Having this polynomial, the values of y for any other x can be obtained. 
This polynomial is called Lagrange’s interpolation formula. 

Let A,(x) be a polynomial which is zero for all x’s, except x,, for 
which it is unity, that is, 


a Qi RFs 

(x)= 

Phd 3 ee 

This polynomial can be written in factored form as follows: 


A, (x) = A, (x — X_)(x ~ xy) +++ (x - Xe pe - xg) oo &- x,) 


(3.40) 


Note that the (x - x,) term is missing on the right of this equation. 


Setting the value of A, (x,) = 1 in the above, we can solve for A, 
te : 
(x, — Xq)(x, — %) +++ (x, — x, (x, — x, 4) + , - x,) 


Substituting this expression for A, in Eq. (3.40), we get 


(x — Xg)(x — x4) va (Ke, = x) ee (x - x,) 


Ate) = 
(x, — Xp) (x, — X4) +++ (x, - X,-1)(X, -— X54 4+ +(x, —x,) 


From this we can obtain Ay: Ay, Ag, Ag, --- by setting s=0, 1, 2,.... 
This results in 
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(CRE) Meee op) oct out 
An(aae 1 2 n 
2 Gp =n Seg ke 
0 ut faes 0} 2 0 n 
(Kimut pp (G— dia) (xe — x) 
Ay (x) re 0 2 n 
(x4 — Xq) (xy - XQ) +++ Oy - x,) 
(3.41) 
(x — xg)(x — xy) -++ (x - 4; _ 4) - X54 4) ++  - ,) 
Aj(x) = 
(xj — Xq) my = xy) +++ Oe; — HH; — Hj) + OE - %) 
(x — Xp) (x — xy) ++ (x - x _) 
A, (x) = S “ 
(x, — Xp), — Hiei Cp Xe, 4) 


Note that in the ith equation (i= 1, 2, 3,...) the (x ~ x,) term is missing 
in the numerator. Now we form the polynomial P(x) using the coeffi- 
cients given in Eq. (3.41) 


n 
P(x) = ¥ 4,9, = Ay (x) ¥q + Ay(xdyy + oe. + A, (x) ¥, 
s=0 


(3.42) 


Using the relation A, = 0 for s # k and A, = 1 for s = k, we get 


2 
P(x,) = 3 Ag %,)¥_ = Any) In = Ve 
s=0 


which is the desired relation for the value of the polynomial at x,. 
Thus, P(x) passes through all points (x,, y,) where k = 0, 1, 2,...and 
the polynomial given by Eq. (3.42) is the desired Lagrange interpolation 
formula for unequal intervals in x. 

The coefficients given by Eq. (3.41) are considerably simplified for 
equal intervals of the variable x. To obtain these relations consider 
the following equations: 
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X = Xp + uh (3:43) 


where s is an integer, uw can be a fraction, and h is the interval between 
two consecutive x’s. Using these relations the coefficients of the poly- 
nomial in Eq. (3.42) become 


yn (u = 1)(u — 2) «+» (u — n) 


Ag(x) = (-1 = 
Ay(x) = (pn- 1 wu = 2)(u - 3) ++. (u— n) 
1!(m - 1)! 
-,udu-I(u-2)... fu-G-D -~(i+D] «+. (u-n) 
Aj) = CD" julu u lus i aM i ] un 
it(n — i)! 
A,(x) = u(u — 1)(u — 2) a [u — (n — 1) (3.44) 
n: 


Note that in the above relations s and wu refer to the position of x, and x 
with respect to x9 as given by Eq. (3.43). 

Example. As an example of the application of Lagrange’s interpolation 
formulas, consider the evaluation sinx for x = 1.2 radians, given the 
following values 


Interval x 
h radians sinx 


%) = 1.00 y = 0.84147 
0.10 nee a o 
x, = 110  y, = 0.89121 


0.20 a yee 


%_ = 1.30 y = 0.96356 
0.10 G 
0.98545 


fd 
oo 
H] 
i 
cs 
So 
S 
) 
" 
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Using the interpolation equations (3.41) for unequal intervals, we have 


(1.2 — 1.1)(1.2 - 1.8)(1.2 - 1.4) A; 
Ay(x) = = 
(1.0 - 1.1)(1.0 - 1.3)(1.0 - 1.4) 6 
m (1.2 - 1.0)(1.2 — 1.8)(1.2 - 1.4) 2 
1 (x) = == 
(1.1 - 1.0)(1.1 - 1.3)(1.1 - 1.4) 3 
A (2:2) — 10) 1. 2)— 1.1)(1.2 = 1-4) _ 2 
g(x) = == 
(1.3 - 1.0)(1.3 - 1.11.3 - 1.4) 3 
A (1.2 - 1.0)(1.2 - 1.1)(1,2 - 1.3) 1 
g(x) = a ee 
(1.4 - 1.0)(1.4 - 1.1)(1.4 - 1.3) 6 
Using these values in Eq. (3.42) we get 
y= SiNX|ap x = 1.2 
1 2 2 1 


Sede . Ae + = (0.96356) — pe 


= 0.93203 


The corresponding value of y given in the trigonometric table is y = 
0.93204, 


Inverse Interpolation 


The Lagrange interpolation formula can be used for inverse interpola- 
tion where the values of ¥p; Yj, 9: +++» ¥, and the corresponding values 
of Xo, xy, X9,..-, X, are given and it is desired to find the value of x for 
a specified value of y. Since in most of these cases the values of y are 
not in equal intervals, Lagrange’s formula for unequal intervals is used. 
The procedure is the same as outlined previously. Note that the Gregory- 
Newton and Sterling formulas given in Chapter 5 can also be used for 
interpolation and inverse interpolation. 


METHOD OF LEAST SQUARES 


The method of least squares is used to find the closest (in a least- 
squares sense) polynomial approximation to a given set of data. Assume 
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that a set of data consisting of m + 1 points 


(x9, Yo)» (x45 Yy)> qr Yq)s 2+ Kas Yq) (3.45) 


is given. We assume a polynomial of degree n < m 
y = P(x) = Ajx® + AL yxt ht oe + Aye t Ay (3.46) 


and evaluate the coefficients A, through Ay by a least-square fit. For 
this we write the expression for the sum of the squares of the errors in y 
evaluated by using the assumed polynomial and the given data. That is, 


The values of A, through A, can be obtained by minimizing the value of 
error E with respect to the coefficients A,, A, _4,---, Ao. This is done 
by setting the partial derivatives of E with respect to these coefficients 
equal to zero 


0E iy 
aA, 
pEiaea 
OA, 4 (3.48) 
OE 
aA, 


This will result in n + 1 equations in n + 1 unknowns A,, through Ao. 
Solution of these equations will give the coefficients of the assumed 
polynomial P(x). 
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y = (9/14)x + 2/7 


@Data points 


figure 3-1. Least squares fit to data points. 
Example. Consider data points (1,1), (3,2), (4,3) to be given in a 


cartesian coordinate system as in Fig. 3-1. It is desired to least-square-fit 
these points by a straight line 


y = Ax+B 


The expression for the sum square error becomes 


Error = E = [1 - (A + B)]2 + (2 - (3A + B)I? + [8 - (4A + B)]? 


The partial derivatives of E with respect to A and B will result 


De Wie BG Sle Bs Ge 4A = BST 
aA 
= -19 + 264 + 8B = 0 
aE 
= -2(1~ A- B)- 2@- 8A- B)- 23 - 4A B) = 0 


= -6+ 8A + 3B = 0 
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or 


26A + 8B 
8A + 3B = 6 


" 
= 
=) 


Solving the above equation for A and B, we obtain the coefficients of the 
required polynomial 


(eo @) 


Given the data points expressed by Eq. (3.45) and the form of the 
polynomial (3.46), a general expression can be derived for the coefficients 
of the set of simultaneous equations with unknown A, ~ A: The 
simultaneous equations are obtained by writing the expressions for the 
derivatives of Eq. (3.48) and collecting terms. For example, the partial 
derivative dE/dA,, will result 


Gathering terms we obtain the first equation in the set of simultaneous 
equations for A, - Ao» 


A, ya Gs x42" Sans za 


rs WAS bre © Page nes ae ee | 


(3.50) 
forse t Ag [xo” + x4" pores $ ea 


ss Yo%o” + ¥%," treet YK 
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The rest of equations can be obtained in a similar manner by setting the 
derivatives dE/dA,,_, --- dE/dAy of Eq. (3.48) equal to zero. The set of 
simultaneous equations for A,, - A, can then be written 


A, (2n] + A,_,(2n - 1] +--+. + Aglal = [y(n] 
A, [2n = A, - [2n many) Yar cree a Agln - U = fym-1)] (3.51) 


A, (i) + A,_,[m - +--+ + Agl0l = LO 


where the notation 


dv - ul (3.52) 


and 


>», xJ = ty] 
i=0 


is used. Denoting the matrix of the coefficients of A’s by the square 
matrix S and the right side of Eq. (3.51) by the column matrix T, we get 


SA = T (3.53) 


where A denotes the column matrix of the coefficients A, - Ao: This 
matrix can be obtained by premultiplying Eq. (3.53) by the inverse of 
the S matrix, that is, 


Ace Set 


The inverse matrix can be obtained, for example, by the augmented 
matrix method described in Chap. 1. 
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COMPUTER PROGRAM FOR LEAST- 
SQUARES METHOD 


The following computer program has been written in FORTRAN (see 
[5]) programming language to implement the calculation of the coeffi- 
cients in the least-squares polynomial fit. Statements 100 and 110 
reserve computer storage space for the following variables. 


Program Symbol Test Symbol Description 

X mf Data points 

¥. y Data points 

CG Ui] Eq. (3.52) 

B (y()] Eq. (3.52) 

s Ss Matrix of the coefficients, 
Eq. (3.53) 

T P. Eq. (3.53) 

A A Matrix of the coefficients, 
A, - Ay 

cy The values of the variable 


y computed from the 
fitted polynomial 
N Degree of the polynomial 
M The number of data points 


Statement 120 reads the degree of the polynomial to be fitted N and 
the number of data points M. Statement 130 decreases the number of 
data points by one to compensate for starting from Xq1 Yor Xp Yys oes 
and so on. Statement 140 reads the coordinates of the data points to be 
fitted. Statements 150-250 compute the values of the coefficients given 
by Eq. (3.52), C’s and B’s of the program. Note that the symbol t 
specifies power. Statements 260-340 set these values in the matrix 
form of the § matrix of the coefficients. Note here that since the 
elements of the matrix start from the first row and the first column, the 
number N representing the degree of the polynomial is incremented by 
unity in statement 260. Statements 350 through 650 are exactly the 
same as the one given in Chap. 1 for augmented matrix inversion method. 
At statement 660 the inverse of the $ matrix will be calculated and 
available. Referring to the augmented matrix method and subsequent 
program of Chap. 1, we see that the inverse matrix is stored in columns 
NX = M+ 1 and NY = 2N rather than 1 to N. This fact is used in state- 
ments 660 through 700 in calculating the product S~! T which represents 
the coefficients A, — Ay. Statement 710 prints these coefficients. 


100 
110 
120 
130 
140 
150 
160 
170 
180 
190 
200 
210 
220 
230 
240 
250 
260 
270 
280 
290 
300 
310 
320 
330 
340 
350 
360 
370 
380 
390 
400 
410 
420 
430 
440 
450 
460 
470 
480 
490 
500 
510 
520 
530 
540 
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DIMENSION X(25),¥(25),C(25),B(25),S(25,25) T(25),A(25) 
DIMENSION CY(25) 
READ, NM 

M=M-1 

READ, (X(I),¥(I),1=0,M) 
NY=2*N 

DO 10 I=0,NY 

C(I)=0. 

DO 10 J=0,M 
C(I)=C(I)+X(N) TI 

10: 

DO 20 1=0,N 

B(I)=0. 

DO 20 J=0,M 
B(I)=B(I)+¥ (J) *(X(U) TH) 
20: 

N=N+1 

DO 30 J=1,N 

DO 30 I=1,N 

NI=2*N-I-J 

S(1,J)=C(N1) 

30: 

DO 35 I=1,N 

T(1)=B(N-I) 

35: 


NY=2*N 

DO 40 J=NX,NY 
S(1J)=1. 

1=1+1 

40: 

L=1 

K=2 

42: XM=S(L,L) 
DO 45 J=L,NY 
S(L,J)=S(LJ)/XM 
45: 

DO 50 I=K,N 
XZ=S(I,L) 

DO 50 J=L,NY 
S(1,J)=S(1,J)-S(LJ)*XZ 
50: 

L=L+1 

K=K+1 
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550 IF (L-N) 42,42,52 

560 52: L=N 

570 53: LZ=L-1 

580 DO 60 K=1, LZ 

590 I=L-K 

600 YZ=S(I,L) 

610 DO 60J=L,NY 

620 S(I,J)=S(I,J)-S(L,J)"¥Z 
630 60: 

640 L=L-1 

650 IF(L-1)65,65,53 

660 65: DO 701=1,N 

670 A(l)=0. 

680 DO 70J=NX,NY 

690 A(1)=S(1,J)*T(J-N)+A\(1) 
700 70: 

710 PRINT,(A(I),1=1,N) 
720 PRINT, tttttt 

730 DO 80 J=0,M 

740 CY (J)=0. 

750 DO 75 I=1,N 

760 CY(J)=CY(J)+A(1)*(X(J) t (N-1)) 
770 75: 

780 80: PRINT,X(J),¥(J),CY (J),CY (J)-¥ (J) 
790 $DATA 

800 2,3 

810 1,1,3,2,4,3 

820 END 


Statements 730 through 780 calculate the values of y at data points 
X9 — Xp,» using the polynomial coefficients just developed. The results 
of the computer run for the case of the previously given example are 
shown below. 

Coefficients for a first-degree polynomial 


A, = 0.6429 A, = 0.2857 


Error 
x Yaata Yeomp Yeomp ~ Yaata 
1.00 1,00 0.9286 —0.0714 
3.00 2.00 2.2143 0.2143 


4.00 3.00 2.8571 —0.1429 
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The coefficients of the polynomial fit can equally well be computed 
for a second-degree polynomial. In this case we have 
Coefficients for a second-degree polynomial 


Ag = 0.1667 A, = -0.1667 Ap = 1.00 


and 
Error 
tg Yaata Yeomp Yeomp ~ Yaata 
1.00 1.00 1.00 0.00 
3.00 2.00 2.00 0.00 
4.00 3.00 3.00 0.00 


This result is expected, since a second-degree polynomial will pass 
through the three data points. 


PROBLEMS 

1. Determine the number of positive and negative roots of the equation 
x5 — 3x4 4 4x3 4 3x2 4 6x + 3 = 0 

2. Find the value of the function 
f(x) = x4 + 6x3 + 8x + 12 
at x = 3 and 5 by synthetic division. Check your solution by 
substitution in f(x). 

3. Determine the polynomial each of whose roots is diminished by 
2 from the roots of the polynomial 
x8 - 8x2 4 19x + 12 = 0 

4. Find the root of the polynomial 
x® — 5x2 4 6x -1 = 0 


between 1 and 2 by Horner’s method. Answer: 1.555. 
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5. Find the root of the equation 


by Newton’s method. 

6. Find the roots of the equation given in Prob. 3 by Graffe’s 
method. 

7. Find the roots of the equation 


P= j 
x3 _ 5y? 4 8x - 6 = 0 Answer: |i. j 
3 
by Graffe’s method. 
8. Find the roots of equation 
x? = 6x7 46x -1 = 0 Answers: 3.2469, 1.5549, 
.19806 
by Graffe’s method. 
9. Find the roots of the equation 
x3 — 2x2 - 8x +5 = 0 Answer: —2.3375, 3.7702, .5674 


by Lin-Bairstow’s method. 
10. Find the roots of the equation 


x3 4 94x2 - 575x + 2,500 = 0 Answer: -100, 3 + 4j 


by Lin-Bairstow’s method. 
11. Find the real roots of the equations of Probs. 9 and 10 by Newton’s 


method. 

12. Given the table of values 

n Vn 

150 12.24745 
152 12.32883_|4_ 
154 12.40967 [1 
156 12.49000_} | p 
158 12.56981 | 


160 12.64911___ | 


13. 


14. 


15. 
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find VI55 using Lagrange’s interpolation formula for equal intervals 
using the A and B sets of values, respectively. 


Using the table of values 
n Wn 

50 3.684 

52 3.732 

54 3.779 

56 3.825 


find the value of n given Wn = 3.756 using Lagrange’s interpolation 
formula (inverse interpolation). Answer: 53. 
Given the table of values 


wn ole 


find the coefficients of the equation y = Ax? + Bx + C fora least- 
square fit to the given points. Answer: y = 3x2 + 2x + 6. 

Find the coefficients of the equation y = Ax? + Bx + C for a 
least-square fit of the table of Prob. 13. Assume that x = n and 
y = Vn. Answer; y = ~.000125 x” + .03675 x + 2.159 


16. Verify the terms of Eq. (3.51) for n = 2 and m = 3, 
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4 TIME-FREQUENCY 
DOMAIN ANALYSES AND 
THE FAST FOURIER 
TRANSFORM 


In engineering analyses, parameters of system behavior are either 
expressed as a function of time or as a function of the frequency 
contents. For example, the solution of differential equations of motion 
of rigid bodies will result in the values of displacements, velocities, and 
accelerations as a function of time, while oscillations of electrical and 
mechanical systems and doppler shifts of radar return are expressed in 
the frequency domain. Time and frequency domain analyses are related 
through Fourier series and transforms. Given a variable expressed as a 
function of time, Fourier analysis will decompose the variable into a 
sum of oscillatory functions, each having a specific frequency. These 
frequencies, with their corresponding amplitudes and phase angles, con- 
stitute the frequency contents of the original variable. 
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A newly developed technique of Fourier transform analyses [1] for 
digital computers, called the fast Fourier transform method, has made 
the use of these computers in frequency domain analyses very attractive. 
The method has been available in mathematical literature for some time 
[2] but it has been rediscovered by Cooley and Tukey only recently 
{3]. The Cooley-Tukey method reduces the number of multiplications 
required in calculating the frequency contents of a time function by a 
factor of y/N, as compared to the direct method of computation, where 
N = 2” is the number of discrete frequencies examined and y is an 
integer. The Cooley-Tukey method is primarily based on using binary 
logic for computation and for this reason the number of discrete 
frequency steps are taken in powers of two. 

In this chapter we will discuss the discrete Fourier analysis using 
matrices for computations, and we will proceed to describe the Cooley- 
Tukey computational process. The report also includes a digital com- 
puter program for obtaining the frequency contents of time functions. 


DISCRETE FOURIER TRANSFORM 


The exponential form of Fourier series of the periodic function F(t), 
with period T, is expressed as 


8 


F(t) = guginne (4.1) 
n=00 
and 
1 1/2 ; 
(hoe bs, F(t)e J"?! dt (4.2) 
T ¥-1/2 
where 


= angular frequency = 2zf 
f = frequency = 1/T 
T = period 


jj PT 
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In Eqs. (4.1) and (4.2) it is assumed that the function is periodic with 
period T and the coefficient a, represents the integral of the function 
over a complete period. The corresponding Fourier transform pair of 
the function F(t) is expressed 


F(t) = 3 Go) el®* do (4.3) 
and 
Go) - Lf Fwett a (4.4) 
27 J-00 


where G(«) is the exponential Fourier transform of F(t). Equation (4.4) 
of the Fourier transform can be compared to Eq. (4.2) of the Fourier 
series if one assumes the period of the function F(t) to extend over the 
entire time domain. In practice, however, the function F(t) is given 
starting from time zero and extending over a span of time T, as given in 
Fig. 4-1(a). This will set the limits of Eq. (4.4) between zero and T. 
Given the Fourier transform G(w) of the function F(t), Eq. (4.3), referred 
to as the inversion integral, will result in the function F(t). For the 
discrete representation of Eqs. (4.3) and (4.4), consider Fig. 41. The 
time span 7' of Fig. 4-1(a) is divided into K equal increments of At each. 
For convenience and efficiency of calculations, let us set K = 2%, where 
y is an integer. The angular frequency span wy is divided into N equal 
increments of Aw each. Denote N in the binary form N = 2%*%1 where 
y,is an integer. Note that although this definition allows the values of N 
and K to be different, in practice they are taken equal. The frequency 
interval Aw is defined as 


1 _ 2nk 
T (N/K) NT 


Aw = 27 Af = (4.5) 


In the above, the value of Af is set at K/NT. This is needed to simplify 
the method of calculation. Equations (4.3) and (4.4) with the above 
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F (t) 
T. 
(a) 
t 
19. 7) to ¥3 t4 tk-3 tK-2 tK-1 tK 
G@) 
ie 
@ 
@9 1 @2 @3 W4 @N-3@N-2@N-] ON 


figure 4-1, Discrete representation of F(t) and G (cw). 


notation can be written in discrete form as follows 


real, 
F(t,) = Aw > Glo,) el@n tk 


K-1 


Glo) = MY Pireiente 


7 k=0 


(4.6) 


(4.7) 


where the limits of the first sum instead of -N/2 to N/2 (corresponding 
to Eq. (4.3)) are set from 0 to N — 1 for computational convenience. 
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As will be seen later, this changes the foldover frequency from zero to 
@y/g- The limits of Eq. (4.7) are taken to represent the available range 
of the variable F(t). Equations (4.6) and (4.7) represent the areas under 
the curves of Fig. 4-1. Using the following definitions, 


F, = Fit,) 
n = Gon) 
QnnK (4.8) 
o, = nAw = 
NT 
t, = RAt = Lae 


Eqs. (4.6) and (4.7) can be written 


N=1 
Fi, = do 3 G,el2nd/N)ink) (4.9) 
n=0 
Ga me a F¥, et-2nd/N) (nk) (4.10) 
7 


Denoting W = e~27//N, incorporating the constant T/2rK into the func- 
tion F,,, and denoting the product as F,, Eq. (4.10) can be written in 
matrix form as 


{G wink) Ce pel OT, 22. N= 2 
Jel ] 
ji naa ual arg Ker (4.11) 


(nx Din x k)(k x D 


where subscripts n and k represent the row numbers of column matrices 
{G] and [F], respectively, and the product nk in addition to being the 
power of W represents the row n and column k location of the W”* ele- 
ment, Before discussing the solution of matrix equation (4.11), con- 
sider the exponential function 
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Wi = ef-27i/N)I _ gos 2mr 1 -— jsin a 1 (4.12) 
N N 


where / = nk is the exponent. Since the exponential function of Eq. 
(4.12) repeats itself in multiples of N, the values of / can be set 


l= nk 
(4,13) 
= nk modulus N 
= integer remainder after division of nk by N 
Under these conditions, / will assume values 0,1,2,3,..., N. It is further 
noted from Eq. (4.12) that 
wo - WN . 1 (4.14) 


and 


WN aie pat eae ye 


As an example, let us take the value of N = 8 = 23 and calculate the 
values of W! where 7=0,1,2,..., 8. Using the relation 


We = e(-2mj/8)1 _ cos(*)- isn(@) dy 10 eAbes aay Bie C4al) 
4 


From this, the values of W! can be calculated and plotted on the com- 
plex plane as shown in Fig. 4-2. It is noted that the foldover frequency, 
where W! and W-! are symmetrically located about the real axis, occurs 
at 4 corresponding to the angular frequency of WN/2 or W4. The values 
of W'(1>.N/2) are therefore the complex conjugates of the values of 
Wire 


MATRIX CALCULATION OF 


FOURIER TRANSFORMS 


In order to effectively explain the logic of calculating the values of 
the matrix equation (4.11), let us set N = K = 8 and write the expanded 
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Imaginary 
axis j 


w2 


figure 4-2. Powers of the exponential function W. 


form of this equation as follows: 


YA oaanp © YW H Of FO 


~— 0 1 2 3 4 5 6 ij 
R 


WOCO,0) yyOCO,1) yyOC0,2) yOKO,3) yOC0,4) yOL0,5) yOC0,6) yy0(0,7) 
wOULO) wlCl.1) w2C,2) w3C1,3) wAC.4) w5C,5) yeU,6) y7,7) 
W020) w22,1) wA(2,2) y(2,3) O24) w2(2,5) wAl2,6) y6(2,7) 
WOC3,0) w3(3,1) y6(3,2) y1(3,3) wpA(3.4) w73,5) y2(3,6) y(3,7) 
WOC4,0) pyAC4,1) yyOC4,2) yy44,3) yyO(4,4) 4,5) yOl4,6) (4,7) 
WOU5,0) py5(5,1) 25,2) w7(5,3) wAl5,4) 15,5) yOt5,6) w3(5,7) 
WO6,0) 66,1) pACG,2) yy26,3) yOIG,4) yEC6,5) yAl6,6) w2(6,7) 


WO(7,0) w7(7,1) yp6(7,2) y5(7,3) wAl7.4) w(7,5) y2(7,6) wl1(7,7) 


Real axis 
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where the power of W, given outside the parentheses, is computed using 
relation (4.13). The row number n and the column number & are given 
within the parentheses (n,k). It is also noted that the W matrix is a 
symmetric matrix and its elements, calculated from Eq. (4.15), are com- 
plex numbers. 

Using the usual method of matrix multiplication, the G matrix can 
be calculated by multiplying the rows of the W matrix by the column 
of the F matrix. This will take N? multiplications, or in this case, 64 
multiplications. In the following discussion, it will be shown that the 
number of multiplications can be considerably reduced if one uses some 
of the properties of the W matrix. The product of W and F matrices 
can be written as 


Column 0 1 ?) 7 
Gy = Fy + WO, + WOR, +... + WOR, 
G, = Fo + WIR, + WF. + --. + WTF, 
G, = Fy+ WF, ‘i WF, eo ee WSF, 
Gy = Fy + WF, + WSF, + --. + WOR, (4.17) 
G, = Fy + WSF, + WOR, +... + WAP, 
G, = Fy + WF, + W2F, +--+ WF, 
Gg = Fy + WSF, + WF, + --. + WF, 
G, = Fy+ WF, + Wer, +... + WIP, 


where W° of the first column of the W matrix of Eq. (4.16) is substituted 
by unity. The computation of G)G, ... G, of Eq. (4.17) can be ac- 
complished by initially setting their values equal to Fy, zeroth column, 
and adding the elements of the first column, and then the second 
column, and so on. It is noted that all of the W’s of the first column 
are multiplied by F',, and those of the second column by Fy, and so on. 
We further observe that in computing the elements WOR, Wry, 
W2F,,..., of the first column, the value of w/2 = W4 is equal to 
—w0 — _1, and the values of W after WN/2 are complex conjugate of 
previously computed values symmetrically located with respect to WN/2. 
For example, we only have to compute 


Wor, Wir, W2F, and WSF, 
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since the rest of the values can be obtained from these, i.e., 


5 2 
Wr, =-WOF, WF, = WF, WSF, = WF, (4.18) 


1 
and WTF, = WIF, 


where bars denote complex conjugates. 

From the second column of Eq. (4.17), it is noted that the values of 
the products repeat after the second Ww Fy is encountered. Thus, in this 
case, we only have to compute the values of wo Fy and WF, since 


WF, = -WOF, and WSF, = WF, 


and the rest of the values W°F,, W2F,, W4F,, and WSF, are merely a 
repetition of the previously calculated set of products. The above 
procedure can be summarized into the following set of computational 
tules: 


1. Denote the number of time increments by K, frequency incre- 
ments by N, and the total time span by T. 

2. Obtain the values of W! where / = 0,1,2,..., N, using Eq. (4.15). 

3. Set the values of Gp, G,,---, Gy_} each equal to Fy (Eq. (4.17), 
zeroth column). 

4. Compute the value of J, nk modulus N, where n is the row number 
and k is the column number of the elements of the W matrix of Eq. 
(4.16), starting with k = 1 and n = 0,1,2,..., N—1. Multiply the cor- 
responding W by F,. The first / will always be zero as it corresponds to 
the power of W in the zeroth row of the matrix of Eq. (4.16). When 
1 = N/2 obtain the complex conjugates of the previously calculated WF 
products, symmetrically located with respect to WN/2. If 1=0 the rest of 
the values of WF repeat. 

5. Add the values computed in step (4) to the existing values of 
G) Gy, ~-+5 Gy _1- 

6. Repeat steps (4) and (5) for k = 2,3,4...,N—1. 


The logic of the above procedure can be programmed for digital 
computer solution. Once such program is given following the numerical 
examples. 
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The values of G calculated above will be complex numbers. The 
element G will correspond to angular frequency w = 0, G, tow = n Aw, 
and so on. It will be shown in the following discussion that the values of 
G symmetrically located with respect to Gy/.(= G, for N = 8) in the 
column matrix of Eq. (4.16) will be complex conjugates, that is, 
G, = G3, G, = Gy and G, = Ge This fact will eliminate calculation of 
G’s beyond Gy/5. 

Consider the values of G, and G, calculated from matrix equation 
(4.16) and given below: 


2 6 
G, = Wor, + WF, + W4F, + WF, + WOR, 


+ WF, + WAP, + WOR, 
ing (4.19) 
G, = WOR, + Wor, + WF, + WPF, + WOR, 


+ WSR. + WAP, + W2F, 


Since W9 - -W4= 1, the underlined values in the above equation will be 
real. From Fig. 4-2 it is observed that W? and W® are complex conjugates. 
Using this fact in Eq. (4.19) will show that G, and G, are also complex 
conjugates. 

For future discussion, denote 


where a, and b, are the real and imaginary parts of G,. The trigono- 
metric and the exponential forms of G,, can be written as follows 


= |G, (cosy, + j sinw,,) 


= [one 


|G, | = absolute value of G, 


= Va,? 7 b,? 
b 
are tan — 


q, 


— 
tl 
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The value of G, can be substituted for G(o,) in Eq. (4.6) to obtain 
the inverse Fourier transform. 


Example 4-1 


As the first example of obtaining the frequency contents by Fourier 
transform, consider the wave of Fig. 43(a). Setting the time and 
frequency steps at K = 16 and N = 16, the F matrix of Eq. (4.16) can be 
calculated by multiplying F (i) by T/27K at each of the time increments 
At = T/K = 1. The sixteen values of W”* are then calculated using Eq. 
(4.12). From Eq. (4.5) the value of Aw = 27/16 = 0.39269. Having 
these values and using the logic described above, the values of G, through 
G,,, can be calculated. 


F(t) 


Time t 


figure 4-3(a). F(t) vs. time, Example 1. 


Table 4-1 gives a list of the values calculated by the digital computer 
program to be described. The values of the function F(t) are shown in 
the first part of Table 4-1 while their Fourier transforms are shown on 
the lower part of the table. The fold over occurs at Gy yg = Gg cor- 
responding to w = 7. It is further observed that the values of G’s sym- 
metrically located with respect to Gg are complex conjugates. 

For the purpose of discussion, let us write the Fourier series [4] of the 
time function of Fig. 4-3(a) 


F(t) = a (ones ae sin3et + I sin5et — -) (4.20) 


72 32 52 


3.2423 sinwt — 0.3602 sin3et + 0.1297 sin 5et - --- 
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Table 4-1. Solution of Example Problem 4-1 


Function F'(t) 


Time step k 


0 0 0 
1 1 1 
2 2 5 
3 3 3 
4 4 4 
5 5 3 
6 6 2 
7 7 1 
8 8 0 
9 9 -1 
10 2 
1 3 


ARGH 


Imaginary part of 
G 


0 0 0 0 0 
1 0.392699 4.18166 0 —4.18166 
2 0.785398 0 0 0 
3 1.1781 0.515635 0 0.515635 
4 1.5708 0 0 0 
5 1.9635 0.230212 0 —0.230212 
6 2.35619 0 0 0 
am 2.74889 0.165452 0 0.165452 
8 3.14159 0 0 0 
“! 3.53429 0.165452 0 0.165452 
10 3.92699 0 0 0 
11 4.31969 0.230212 0 0.230212 
12 4.71239 0 0 0 
13 5.10509 0.515635 0 —0.515635 
14 5.49779 0 0 0 
5.89049 4.18166 0 4.18166 
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where w = 27/T = 0.392699. Consider the inversion Eq. (4.6) of the 
Fourier transform 


Fi) = Ao}(ay + jbp)[eoswyt + j sina t) 
+ (a, + jb) [cosa t + jsine,¢) (4.21) 
+ (dg + jby) [cos wot + jsinwst] 


4 


where a, and },, as before, are the real and imaginary parts of G,. 
Substituting the values of a and b from Table 4-1, we get 


F(t) = 0.392699} (4.18166) (cosa, t + j sino, 0 
+ (0.515635j) (cosa,t + j sinw,t) 
+ (—0.230212j)(cos@,t + j sinws t) 
+ (0,165452))(cosw7t + j sine, ) (4.22) 
+ (—0,165452j)(cosagt + j Sins t) 
oa (0.230212)) (cos w,;¢ +j sinw,,!) 


oof 


Using the relations ®,, = -1, 13 = -@3, and @,, = —@5, Eq. (4.22) 
simplifies to the form 


F(t) = 3.28419 sinw,t - 0.4050 sin ogt + 0.1808 sinwst - --- 


Comparison of the above and Eq. (4.20) indicates that although the time 
function of Fig. 4-3(a) was represented by only 16 points, the agree- 
ment between Fourier transform and Fourier series is acceptable. 


Example 4-2 


As a second example, consider the function F(t) of Fig. 4-3(b). In 
this case we again set N = K= 16 with T= 16. The results of calculation 
are shown in Table 4-2. The values of G given in this table should be 
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figure 4-3(b). F (t)vs. time, Example 2. 


interpreted in conjunction with the inversion equation (4.6). Using the 
exponential form of G’s 


G, = |G,|evn 


where |G,| is the absolute value of G,, and &, is the phase angle, Eq. 
(4.6) can be written 


N-1 
F(t,) = Aw > |G, | ef (@nte + Yn) (4.23) 
n=0 


The above equation represents the value of the function F(t,) for 
specific values of k. For example, for k = 0, t, = 0, we have 


N-1 
F(0) = Ao y |G, | ein 
n=0 
N-1 
= do ) (a, + jb,) 
n=0 


Substituting the values of a, and b, from Table 4-2, the above sum will 
result in the value of F(0) = 100. 

The amplitude Aw |G,,| of Eq. (4.23) can be plotted as a function of 
as shown in Fig. 4-4, Note that in this figure the foldover frequency 
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Table 4-2. Solution of Example Problem 4-2. 


Time step k | Timet | Function F(t) 


i) 0 100 
1 1 100 
2 2 100 
3 3 100 
4 4 100 
5 5 100 
6 6 100 
7 7 100 
8 8 

9 (4 

10 10 


ARG 


Bie Sor Imaginary part of 


see value of Real part of G 


0 0 127.324 127.324 0 
1 0.392699 81.5801 15.9155 —80.0126 
2 0.785398 0 0 0 
3 1.1781 28.6471 15.9155 23.8192 
4 1.5708 0 0 0 
5 1.9635 19.1414 15.9155 —10.6344 
6 2.35619 0 0 0 
7 2.74889 16.2273 15.9155 — 3.16579 
8 3.14159 0 0 0 
9 3.53429 16.2273 15.9155 3.16579 
10 3.92699 0 0 0 
11 4.31969 19.1414 15.9155 10.6344 
12 4.71239 0 0 0 
13 5.10509 28.6471 15.9155 23.8192 
14 5.49779 0 0 0 
5.89049 81.5801 15.9155 80.0126 
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Amplitude 
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figure 4-4. Amplitude-frequency plot of Fourier Transform of Example 2. 


occurs at @g and the values of @g = -@7, ©; = —w¢, .--, are plotted as 
negative frequencies. 


COMPUTER PROGRAM FOR THE CALCULATION 
OF FOURIER TRANSFORM 


The following computer program has been written in BASIC program- 
ming language [5] to implement the previously discussed calculations. 
The relation between the parameters of the program and those of the 
text are 


Program (symbol) Description Text (symbol) 
K1 Number of time steps K 
N Number of frequency steps N 
SY Time span F 
Tl Time increment At 
R(J) Real part of W/ 

I(J) Imaginary part of WS 
F(J) Time function at j At 
G(J) Real part of G(aj) 
H(J) Imaginary part of G(wj) 
A Absolute value of G 


Nl Foldover value of G N/2 
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The loop 180-210 computes the required values of W. The loop 
220-250 reads into the computer the time function F(t) and incorporates 
the factor T/27K of Eq. (4.10). The loop 260-290 sets the values of G 
equal to Fy. The loop 300-640 is used to compute the modulus of nk by 
N (this can be done more efficiently by binary arithmetic, see foot- 
note) and computes the required values of W multiplied by F in each 
column of matrix W. When the value of L becomes equal to N/2 (or 
N1 in the program) statement 380, the product WN’2F; is set equal to 
-WF., statement 420, and the complex conjugate values of previously 
obtained WF products are calculated, statements 480 and 490, When L 
becomes zero, statement 370, the values of W repeat and the instructions 
are transferred to statement 540. In the following loop 550-630 the 
calculated values of WF are incorporated into proper values of G. At 
the completion of statement 640, all the values of G’s are calculated and 
the rest of the program simply prints these values. 


FFFTTT 16:31 THURS, 04/11/68 


100 DIM R(100),1(100),F(100),G(100),H(100),U(100),V(100) 
110 READ K1,N,T 

120 LET P1=3.1415927 

130 LET P=(P1"2)/N 

140 LET T1=T/K1 

150 LET T2=P/T1 

160 LET N1=N/2 

170 LET T3=T1/(2"P1) 

180 FOR J=OTON 


The calculations of modulus of nk by N as given in statements 320-360 can be considerably 
simplified by using binary arithmetic. For example, for the row number n = 7 and the column 
number k = 6 the product nk will be 42, which in binary arithmetic is written 


5 ea 2 1 0 Power 
32 16 8 4 2 1 Powers of 2 
1 0 1 0 1 oj Product nk in binary bits 
(2+ 8+ 32) 
y =3 bits 


The remainder is y binary bits on the right of the product nk, where y is obtained from 
N=2”. In the above, the remainder is equal to 2. 
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LET R(J)=COS(P*J) 
LET I(J)=-SIN(P*J) 
NEXT J 

FOR J=0 TO K1-1 
READ F(J) 

LET F(J)=F(J)*T3 
NEXT J 

FOR J=0 TO N-1 
LET G(J)=F(0) 

LET H(J)=0 

NEXT J 

FOR K=1TO K1-1 
FOR J=0 TO N-1 
LET L=J*K 

IF L=N THEN 540 
IF L>N THEN 360 
GO TO 380 

LET L=L-INT(L/N)*N 
IF L=0 THEN 540 
IF L=N1 THEN 420 
LET U(J)=R(L)*F(K) 
LET V(J)=H(L) *F(K) 
GO TO 510 

LET U(J)= -U(0) 
LET V(J)= -V(0) 
LET J3=J 

LET J3=J3-1 

IF J3<1 THEN 510 
LET J=J+1 

LET U(J)=U(J3) 
LET V(J)= -V(J3) 
GO TO 450 

NEXT J 

LET J1=J 

GO TO 550 

LET J1=J-1 

FOR J=0 TO N-1 
FOR M=0TO J1 
LET G(J)=G(J)+U(M) 
LET H(J)=H(J)+V(M) 
LET J=4+1 

IF J>N-1 THEN 640 
NEXT M 

LET J=J-1 

NEXT J 
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640 NEXT K 

650 PRINT “TIME STEP”, “TIME”, “FUNCTION” 
660 FOR K=0TO K1-1 

670 PRINT K,K*T1,F(K)/T3 

680 NEXT K 

690 PRINT ; 

700 PRINT “FREQ STEP”,“OMEGA",”“ABS VAL",“REAL PART","IMAG PART” 
710 PRINT 

720 FOR J=0TON-1 

730 LET A=SOR(G(J)t2+H(J)t2) 

740 PRINT J,T2*J,A,G(J),H(J) 

750 NEXT J 

760 DATA 16,16,16 

770 DATA 0,1,2,3,4,3,2,1,0,-1,-2,-3,-4,-3,-2,-1 

780 END 


COOLEY-TUKEY METHOD 


The Cooley-Tukey method of computing the G matrix of Eq. (14.16) 
is based on expressing the W matrix of the same equation in terms of 
products of y square matrices, where y is related to the order of these 
matrices N by N = 2%. In this method the number of time and frequency 
steps are the same and are equal to N. Consider the expanded form of 
Eq. (4.11) for N =4 


Go WO.0 Ot yO.2 3] | RO 
i; 1,1 71,2 y1,3 
Realm ies aie er ce me aca (4.24) 
Gy ~ | w2.0 wl w22 pw23 Fy 
3, 3,1 73,2 73,3 
Gs w3.0 Ww W w3 F, 


Taking the products of the exponents of W, the term nk, and obtaining 
their modulus for N = 4, we get 


Gy WOE peas, PO Sh Fe 
j0 1 2 3 
Rath ae as Ma did (4.25) 
Gy Wo = WPieawoe eWA NIRS 
G3 wo ws w2 wil IF, 
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Noting that the value of W° = 1, the W matrix of the above equation can 
be written as the product of two matrices (y = 2) as follows: 


Go| [2 we 0 ot (2 0 -wo off 


0 
Gy f TWA) 20» 05 )) Oem Toe derr eh nae (4.26) 
G, 0. <0. Wala =o" #2 online 
G, 0 0 2 Wy] Jo a Nokpeti 


The product of the above matrices will result in relation (4.25). Note 
that in Eq. (4.26) the order of the elements of the G matrix is changed. 
The product of the last two matrices of Eq. (4.26) is denoted as [F]! and 
is written 


POS yoine (4.27) 


(Fl! = 
Wes OES 


o 


2 
0 1 0 WI /R, 


From the above, the elements of the [F]! matrix are 


1 

Fy = Fy + Wr, 

Fy = Fy + Wor, (4.28) 
1 72 

Fy = Fy +W Fy 


2 
Fz = F, + W°F, 


Having the elements of the [F]! matrix and using Eq. (4.26), the ele- 
ments of the G matrix are computed. 

The order of the elements of the G matrix in going from Eq. (4.25) 
to Eq. (4.26) is computed using binary arithmetic. The subscripts of 
these elements are first written in binary and then inverted to obtain 
the new order as follows: 
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Original Binary form Symmetrically 


Subscript (y binary bits) inverted form Final subscript 
0 00 00 0 
1 ol 10 2 
2 10 ol 1 
3 11 11 3 


As a further example for N = 8 = 23, the above method will use 3 bit 
binary numbers resulting in the order of G’s as given below: 


Original Binary form Symmetrically : 7 
Subscript (y binary bits) inverted font Final subscript 

0 000 000 0 

1 001 100 4 

2 010 010 2 

3 oll 110 6 

4 100 001 1 

5 101 101 5 

6 110 oll 3 

x 111 111 7 


Multiplications and additions performed in obtaining the elements of 
{F}! matrix can be done by writing a computation tree and subsequently 
using binary arithmetic in computing the logic of operations. 

For the four-by-four W matrix of Eq. (4.26) consider the tree of 
Fig. 45 where the original F matrix is placed on the left. 


Column ——» 0 
Row 


figure 4-5. Computation tree of the Example. 


Solid lines of Fig. 4-5 represent multiplications, and dashed lines repre- 
sent additions. The numbers in small circles are the powers of W. For 
example, F. # is obtained by multiplying F, by W? and adding F, to it, 


178 Digital Computer Methods in Engineering 


as given by Eq. (4.28). The logic of determining the powers of W, which 
are entered in the circles, and the multiplications and additions are done 
using binary arithmetic as given below. 


1. Assume N sampled values of F(t) where N = 2”. The integer y 
will equal the number of bits in the binary representation of N. 
2. Arrange for y columns of [F]!, [F]?, [F]°, ..., [F]” to be calculated. 
3. Let the numbers in the circles representing the powers of W be 
denoted by p. For row k and column / (starting with k and / equal to 
zero), the procedure for computing p is as follows: 
a. Write the binary number k. 
b. Slide this number y - / places to the right filling the newly 
opened locations with zeros. 
c. Reverse the order of the resulting binary bits. The resulting 
integer is p. 
For example, for N = 8 = 23(y = 3) the power in the circle of the 4th 
row (k = 4) and second column (/ = 2) with y — / = 1 is computed as 
follows: 
a. The binary representation of k is 100. 
b. Since y - / = 1, shift k one bit to the right, resulting in the 
binary representation 010. 
c. Reverse the order of (b), 010, which in integer form equals 2, 
that is, p = 2. 
4. For multiplication and division denote the binary representation of 
row k by 
k = hy_1 «+ Roky Ro 
where k, k,,-.. are either zero or one. In column 1, node k has a 
solid line drawn to it from a node in the (J — 1) column having the 
same binary representation as k except the bit location k,_, is replaced 
by one. For N = 16= i (y = 4) the element in the third column (/ = 3), 
fifth row (k = 5), will have a solid line drawn from the element in the 
second column having the row number determined by writing 


k = kgkgk)ky = 5 and inbinary = 0101 


Replacing k, element by 1 we get row number = 0111 = 7, or row 
number seven. The dashed line comes from a node in the (J — 1) 
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column with binary representation of k, where k,_; is replaced by zero. 
In the above example this will result in k of 5. 

Using the above logic the computation tree of Fig. 4-6 for N = 16 can 
be constructed. 


DISCUSSION OF FOURIER TRANSFORMS 


Assuming that the function F(t) is real, which is usually the case, the 
discrete values of its Fourier transform can be obtained by multiplying 
W and F matrices of Eq. (4.16). For N discrete values of Fourier 
transform G, this will take N? multiplications of complex numbers W’s 
by real numbers F’s. Taking advantage of the properties of the W 
matrix of Eq. (4.16) and the arrangement of product terms as given in 
Eq. (4.17), the number of multiplications can be considerably reduced. 
For example, for N = 16, the product matrices of Eq. (4.16) will require 
256 multiplications of complex numbers by real numbers. Using the 
matrix calculations described by the six-step logic will reduce the number 
of multiplications to about 85. Note that this method does not require 
multiplication of complex numbers by complex numbers which account 
for four real multiplications. 

The number of multiplications in the Cooley-Tukey method can be 
shown to be y\ (where N = 2%). By this method for N = 16 (y = 4) the 
number of multiplications will be 64. As can be seen from the compu- 
tation tree of Cooley-Tukey method these multiplications involve com- 
plex numbers by complex numbers which should be counted as four 
real multiplications. 

Since a general expression for the number of multiplications required 
in the matrix method is not available, a quantitative comparison of 
matrix and Cooley-Tukey method cannot be made. It is estimated, 
however, that the real computational advantage of Cooley-Tukey method 
is to be gained when N assumes very large values (in the order of N = 28 
and above). 


PROBLEMS 


1. Show that G, and G, of Eq. (4.17) are complex conjugates, 

2. Obtain the Fourier series of the function F(t) of Fig. 4-3(a). 

3. Obtain the real and imaginary parts of G, and G, of Table 4-1 by 
directly multiplying the proper elements of matrix equation (4.11). 
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Setting N = K = 8, find the set of values Gp,..., G; for F(t) of 
Fig. 4-3(a). Answer: Aw = .392699, |Go| = |Gg| =9, |G,| = 
4.34711, |G3| = .745846 

Repeat the above problem for N = K = 4 and find the values 
Go, G,, Gg, and G3. Answer: Aw = .392699, |Gp| = |Go| = 
0, |G,| = 5.09296 

Obtain the Fourier transform of 


Answer: Fo thru F7 = 2, 2, 2, 


F(t) 2, -2, -2, 
: Aw = .785393 
‘ |G,| = 1.66357 
| Y Timet |G, = 689072 
-2 |Go| = |Gq| = 0 


by matrix method using N = K= 8. 
Solve the above problem for N = K = 4. Answer: |\G,| = 1.80063 
Obtain the Fourier transform of 


Answer: Fy thru F, = 8,8,8, 
F(+) 


8 8,6, 4, 
2,0 
|Go| = 14.0056 
8 16 Time t \G,| = 4.61834 
\G4| = 1.27324 


by matrix method using N = K=8. 
Obtain the Fourier transform of 
Answer: N =K =8, Aw = .785398 
Fe | |G,| = 4.45634... |G,! = 
63662 
N =K =4, Aw = .785398 


. |Go| = 3.81972... |Go| = 
8 Time t 1.27324 2! 


by matrix method using N = K = 8 and compare with N = K= 4. 
Obtain the number of complex multiplications (complex numbers 
by real numbers) required for the computation of the G matrix 
of Eq. (4.17) for N = K= 8 and N = K=4. 
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11. Obtain the Fourier transform of the function of Fig. 4-3(b) for 
N= K=8, 

12. Obtain Eq. (4.25) by performing the matrix multiplication of Eq. 
(4.26). 

13. Verify that the values of Go, ..., G3 obtained from Fig. 4-5 are the 
same as those of Eq. (4.25). 

14. Verify the additions, multiplications, and powers of W in the tree 
graph of Fig. 4-6. 

15. Write a tree graph for N = 8. 

16. Obtain the Fourier transform of Prob. 6 by the Cooley-Tukey 
method for N = K= 4. 

17. Obtain the Fourier transform of Prob. 8 by the Cooley-Tukey 
method for N= K=8. 

18. Obtain the Fourier transform of Prob. 9 by the Cooley-Tukey 
method for N = K=8. 
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5 NUMERICAL 
METHODS AND CALCULUS 
OF FINITE DIFFERENCES 


In the numerical solution of mathematical problems, calculations are 
performed at discrete time or space intervals. The numerical values of the 
solutions are obtained at these intervals. This is to be distinguished from 
classical solutions where a continuous solution in the form of an 
equation is derived and it can be evaluated for the given values of the 
independent variable. To illustrate this, consider the second-order 
linear differential equation 


2 
eter 0 (5.1) 
dt? 


where ¢ is time and the initial conditions are y = 1, dy/dt =0 at 
183 
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t = 0. The solution of this equation is of the form (see Appendix) 


y = Asint + Beost 


where A and B are constants, to be evaluated with initial conditions. 
This results in 


y = cost (5.2) 


This solution gives the value of y for any value of time ¢. On the other 
hand, the numerical solution of differential equation (5.1) is obtained 
by selecting a time interval At transforming the equation into difference 
form (which will be discussed later) and obtaining a recursive equation 
for y in terms of two values of y at preceding time intervals. Or, in 
functional form 


Vier = (Np ¥;-p AD (5.3) 


the indexes i and i— I refer to values of y at times t; and t;_,(t;-t;_; = Ad 
currently available and i + 1 refers to the value of y at t;,, to be 
obtained. Thus, the application of the above relation together with the 
initial conditions will result in values of y at 0, At, 2At, 3At, ... and so 
on. It is important to note that, unlike solution (5.2), the value of y at 
t= nAt (where n is an integer) cannot be obtained unless the previous 
values of y are available. 

It is seen that the numerical calculations involved in obtaining the 
value of y from Eq. (5.3) at a given time ¢ are considerably more than the 
evaluation of solution (5.2). This is true in the above example since an 
analytical closed form solution exists for the differential equation (5.1). 
However, many engineering applications of numerical methods involve 
nonlinear differential equations, polynomial interpolation of tabulated 
functions, and the solution of transcendental equations where no 
analytical closed-form solutions exist. In these cases, solutions are 
obtained by numerical formulation of the problems and application of 
digital computers for evaluation of solutions. 

It should be also pointed out that in most engineering problems, even 
if closed-form solutions are available, they should be evaluated and 
plotted while the numerical methods result in solution already in 
tabulated or graphical forms. 
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Xim2 Mim] Mi R41 Xi42 — 


—2Ax-Ax 0 Ax 2Ax 


figure 5-1. The function y = f(x). 


DIFFERENCE OPERATORS 


In numerical solutions, the difference operators are used in a manner 
similar to that of differential or Laplace operators (usually written as D 
or S) in the solution of differential equations. A complete familiarity 
with the difference operator is considered essential for an understanding 
of the underlying principles of numerical methods. 

In order to define various difference operators, consider the function 
y = f(x) described in Fig. 5-1 with equal intervals Ax for the independent 
variable x. 

The first backward difference of y; is defined as 


Vy = DP— F324 Cas) 


where VY denotes the backward difference operator. 
The second backward difference can be obtained from the first: 
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Vy, = ViVy) = Vy; - ¥4-) 


WW 


oe avared (5.5) 


Q; = 93-4) - O4-1 - 4-2) 


" 


¥;= Min + Vio2 
Similarly, the third backward difference becomes 
V8y; = 9; - 89;-1 + 894-2 - Yi-3 (5.6) 
The first forward difference of y; is defined as 
AY; = %i41- 9: (5.7) 
where A denotes the forward difference operator. The second and third 


forward difference operators can be obtained from the first and second, 
respectively: 


A?y, = AtAy)) 
Ayi41 — AY; 
= %ix2- Wier? i (5.8) 
ASy, = A(A?y;) 


= Visa — Wise + Vier — % 
The first central difference operator of y; is defined as 
8Y; = Vise - Yi-ve2 (5.9) 
where 5 denotes the central difference operator. Note that the first 


central difference involves the values of the function at half intervals. 
The second central difference operator is obtained from the first: 


i 


dy, = 8(8y) = 60;4 2 - Yi-v2) 


O34 ve+ve ~ %ieva-12) — Vi-1a+12 - %i-v2-v2) 


= Veer (5.10) 
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Similarly, third and fourth central difference operators can be obtained: 


9439/2 ~ i412 + Yi-12 - Yi-32 


B8y, = 8(5%y;) 


(5.11) 
i= 5 (5%y,) = 9i49.— We,1 t 9; — Yj-1 + 1-2 

As a numerical example of the above equations, consider the calculation 
of difference operators of the equation y = x? as given in Table 5-1. In 
this table, the second and higher differences are obtained by arithmetical 
operations on the lower differences, for example, V2y - V(Vy) or V2 y = 
5 —3=2. Only the value of 5”y is given since dy will require values of 
y at half interval points (i + 1/2). 


Table 5-1. Difference Operators of y = x” 


ocoooooooo 


KH OWOMIADUNAEWNH 
COMIDNAWNHKO 
NNNNNNNHNNN 
ooooocoeco 
NNNNNNNNN 


RELATIONSHIP BETWEEN DIFFERENCE AND 
DIFFERENTIAL OPERATORS 


Let us define the symbolic difference operator E as follows: 


Ey; = Vi+1 
Ely, byt (5.12) 
1 
Ey; = Yisie 


Using these relations, we can obtain difference operators V, A, and 6 
in terms of E 
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Vy, = ¥4- ¥j-1 = 94 - BY; = ( sa a 


ree N cei tees Oia UI 


= = EV2 -/2y _ (pV/2 _ R-1/2 
Fe aS YP Ress eigen gi MO URI 


Equating both sides of these equations, we have 


Vieol= i backward difference operator 
E 

(5.13) 
A=E-1 forward difference operator 


6 = EV2_E-V2 _ central difference operator 


The following algebraic relations also hold between the operators: 


Ak = kA 
Ek = kE 5.14 
same for V and 6 operators ( ) 

EA = AE 

E" A = AE” 


where k and n are constants. 

The relation between difference operators. discussed above and the 
differential operator is obtained by writing Taylor’s series expansion of 
the function f(x + h)about x. This results in [2] 


, nh? nh” 
f(x +h) = Anat eee te tee +— Go) (5.15) 
. n: 


where primes denote derivatives with respect to x. Denoting the differ- 
ential operator D = d/dx and using the difference operator E, Eq. (5.15) 
becomes 

re) 


h? ne 3 
Ef (x) = [ + AD + Tis - ie pose] f(x). (5.16) 
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Equating both sides of Eq. (5.16) and noting the infinite series in the 
brackets represents the series for the exponential function, we have 


i ehD 
or 
ie ne (5.17) 


The above formulas can be applied to obtain relationships between 
difference and differential operators. These relationships are used in 
the numerical solution of differential equations, interpolation and 
numerical differentiation, and integration of tabular values. 


RELATIONSHIP BETWEEN VY AND D 


The backward difference operator VY can be obtained in terms of the 
differential operator D with the aid of Eq. (5.13): 


(5.18) 


i 
= 
S 
! 
| 
io] 
1S) 


Similar expressions can be obtained for higher-order difference opera- 
tors: 
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V2 = (1 — e“hD)? _ 1 4 e-2hD _ 9g-hD 


2( nD h2D2 733 ) 
ry | ae = rear 


= np? — n3p3 , 1 44 D4 — 
2 


The product of Eqs. (5.18) and (5.19) will give the value of V3 in terms 
of differential operator D. 

Conversely, the values of differential operator D can be obtained in 
terms of the difference operator V by using the relation 


eAD 2 1-V 


(5.20) 
Ine“?D — _hD = In(1 - VY) 
Writing the series for In(1 — VY), we have 
2 3 4 
p= 3.2.2.2...) (5.21) 
h 2 3 4 
Higher-order derivative operators D?, D®,..., can be obtained suc- 


cessively by forming products of lower-order derivatives. 
The relationships between V and D operators are summarized below: 


2 3 
ay ene Le eae (5.22a) 
2! 3! 
v2 — h2p2 — A3p3 4 A4p4 _... (5.22b) 
12 
3 5 
3 _ _3p3 4p4 5 pd 
Vv esa ree (5.22c) 
1 Vee evs 
o-10.8,8 4... (5.224) 
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D2 = 1 (ve. v8, Hos, Svs...) (5:22e) 
) 12 6 

D3 = 1 (oe. Su, Zoe. .) (5.22f) 
1B 2 4 

Dace 1 (os + ov, lye , +) (5.228) 
nt 6 


The above relations give the infinite series representations of backward 
differences in terms of differentials and vice versa. In using these equa- 
tions in actual computation, however, we can only take a finite number 
of terms. Assuming that the series are convergent and that each term 
is smaller in magnitude than the preceding term (this is usually the case 
since the increment ’ is small and its powers increase), the first term 
dropped will represent the largest term neglected in the succeeding cal- 
culations. This term will give the “order of magnitude” of terms 
neglected in the finite difference representation. It should be noted that 
in truncating an infinite series the sum of all of the terms dropped gives 
the actual truncation error. Since getting an expression for this sum is a 
formidable task in many cases, the first term dropped from the series 
is taken as the “order of magnitude” of the error introduced. For 
example, taking the first two terms of Eq. (5.22b) and neglecting the 
terms 42h*D* and beyond will result in an error equivalent to the 
absolute value of this term. In addition, as will be seen in the discussions 
of this and the following chapters, since in many problems the value 
of the derivative of the variable D*y is also an unknown, this absolute 
value cannot be computed. For this reason we make another simplifica- 
tion in describing the error by stating that neglecting the third term and 
beyond of the series of Eq. (5.22b) introduces an “error of order nt” in 
the resulting expression for V2. The error of order h* is written 
symbolically as error of 0(h4). 

From Eq. (5.22) the derivatives of various functions can be obtained 
in terms of the available values of the function. Suppose that we want 
to obtain the value of D in terms of V’s with error of order h. From 
Eq. (5.22a) V2 involves terms of A? and higher and since in Eq. (5.22d) 
this term, V2, is divided by h, neglecting it in Eq. (5.22d) will result in an 
error of orderh, as follows 
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Di= wie Dy; = lyy, 
h h 
(5.23) 
B Tamed il 
se Pe 04,9 
h 


where e = O(h) refers to the error of order h. If the error is to be of 
order h2[e = O(h?)], the V2 term in Eq. (5.22d) should be retained and 
the V3, V4, ... terms dropped. This, using definitions of V, V2 results 


in 
1 v2 
Bre ( : a 


1 2 
Thal gig + Y;-9) e@ = O(h*) 


" 


(5.24) 


This result can also be obtained by setting V = 1 — E~! from Eq. 
(5.13) and raising to appropriate powers, that is, 


1 v2 
Dy; = 1( ay es, 


The above relations can also be obtained for higher-order differentials. 
Table 5-2 summarizes the results. 


RELATIONSHIP BETWEEN A AND D 


The equations relating the forward difference operator A and the 
differential operator D are derived in a similar fashion as before. In 
this case from Egs. (5.13) and (5.17), we have 
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Table 5-2. Relationship between Backward Difference Operator V and 
Differential Operator D 


B= 14 A = ef? 
(5.25) 
Die in eens 
h 
resulting in 
2 3 
hy em Ties aera oy er a Pace 
2! 3! 
2 3 
epee apse 
2! 3! 
and 
Cea: 4 
De ee ee (5.26) 
h et tee 


Higher-order derivative and difference operators are obtained by 
taking various powers of Eq. (5.25). This results in the following second- 
and higher-order operator equations: 


194 Digital Computer Methods in Engineering 


ag yg oUanear a ahr 9p ts (5.27a) 
12 

A3 — 1303, 2n4p4 « 2n8DP 4 ... (5.27b) 
2 4 

1 11 5 

pe. L(s2 a8 Hat Sas...) (5.27c) 

felt fy MCLE Bea rata (5.27d) 
De = 2 (A8— 2 At OA 
ne 2 4 

D4 = 1 (a4 = 2A5 4, iy ~ 2 (5.27e) 


Equations (5.26) and (5.27) can be used to find the differentials of 
functions in terms of available functional values. For example, neglecting 
errors of order h and higher, D?y, can be obtained from Eq. (5.27c). 
Dropping A® and higher-order terms from this equation and using Eq. 
(5.8), we have 


D?y. = —A*y 


(5.28) 


Using this procedure a table of coefficients, corresponding to Table 5-2, 
can be calculated for forward-difference operators. This is given in 
Table 5-3. Note that the second relation above can be derived from the 
first by substituting A = 1 — E from Eq. (5.13) and using the definition 
of difference operator, that is, 


D2y, = LE = 1)?y, 
ar 


- +a - 2 + By, 


n2 


1 
z rh. — 29541 + Yix9) 
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Table 5-3. Relationship between Forward Difference Operator A and 
Differential Operator D 


RELATIONSHIP BETWEEN 6 AND D 
OPERATORS 


The equations relating the central difference operator 5 and differ- 
ential operator D can be obtained using Eqs. (5.13) and (5.17). 


5 -EV2_p-V2 get? p- ling (5.29) 
h 


In addition, let us define the mean operator »: as follows: 


= 4(E + E-V2) (5.30) 


This will result in 
pd = 2(B - E74) = L(e%P _ ehP) = sinh(hD) 
2 z (5.31) 
p5(y)) = 5441 — i-d 


and 


w= 1+ E)+ 2 = ce, (5.32) 
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Substituting the Taylor series expansion for sinh (hD), Eq. (5.31) becomes 


n3D3 PD? 
+ — + ——— +e 


6 = hD 
: 6 120 


(5.33) 


Using the expression for 5 from Eq. (5.29), squaring and substituting 
E = e” we have 


AD -hD 
2 -1 e +e 
5 -@-2.60 ore) (5.34) 
= 2[cosh(hD) - 1] 


Substituting the series for cosh (kD) results in 


4 p4 6 ps 
Bete Ree te Dee (5.35) 
12 360 
The general expression for 5", where n = 1,2,3,.. ., can be obtained from 
Eq. (5.29) 
8 = ehD/2 _ o-hD/2 _ 9 sinh tD 
2 
(5.36) 
an = 2% sinnn 2D 
2 


Using combinations of the above relations, we can obtain expressions 
for higher powers of central difference operators. For example, combi- 
nation of Eqs. (5.33) and (5.35) gives 


3p3 5 pd 
p83 = (y8)(52) = (io. Bor Dye 2) 


6 120 
D2 4 hips n® pe ee (5.37) 
12 360 
5pd 277 
n3p3 , #D® , kD 
4 40 
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Squaring the first few terms of Eq. (5.35) will result in 


n&p& — AB D8 
——_ + poses 
6 80 


64 = h4D* + 


(5.38) 


The differential operators D can also be obtained in terms of central 
difference operators 6. 
For example, from Eqs. (5.31) and (5.32) we get 


tens 1 53 8° 
b= sinn-tys - 1 (a - 5.39) 
h 4 ‘( 6 30 é 


where the right side of the equation is obtained from Taylor series 
expansion of sinh~',5. Similarly, 


p-1(2_F 5...) (5.40a) 
n2 12 90 

ps. @ Sp a ) (5.40b) 
ne 4 120 
6 8 

n= (-2....) (5.40c) 
nt 6 240 


Note that in the above relations, only odd differentials (D and D®) 
contain terms involving p’s. 

Using the above relations, a table can be obtained relating differential 
and functional values. For example, the second differential D2, neglecting 
e = O(h2), can be obtained from Eq. (5.40a). From Eq. (5.38), 54 
involves terms of h* and higher; since this term (54) in Eq. (5.40a) is 
divided by h?,neglecting this will result in errors of order h? and higher. 
Using Eq. (5.10), this results 


+ (9%)y, = ~O4441 - 2y,+9;-) -e = 02 (5.41) 
h 


D?y; = 
h2 
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This result can also be obtained by substituting the value of 
6 = EV2 _ E-V2 from Eq. (5.13) and using the definition of operator 
E, that is, 


1 


D?y, = —(8%)y, 
h2 
é we ~ E-V2yy, 
1 ; s 
= —(E-2+ Ey; 
h2 


1 ‘ 
= Open 07 F Hien) 
fe 


Table 5-4 gives the relation between various orders of differential 
operators and central difference operators. 

The forward and backward differences are primarily used in incor- 
porating the initial and/or boundary conditions (as will be seen in 
Chaps. 6 and 7) in the finite difference representation of the differ- 
ential equation while the central differences are used for getting the 
general finite difference formulation. 

Example. As an example of application of the above equations, 
consider the numerical solution of the differential equation 


Table 5-4. Relationship between Central Difference Operator 5 and 
Differential Operator D 
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dy dy 
ate vl = 0 
qa dt (5.42) 


Subject to initial conditions, y = 1, dy/dt = 0 at 1 = 0. This equation in 
operator form can be written 


Dy; + Dy; + y; = 0 (5.43) 


where D = d/dt and the initial conditions are yg = 1, Dyg = 0. Using 
the central differences of order h?, from Table 5-4, we have 


¥p-1 -— Wier; 
pe yee ee Oh) 


: r (5.44) 
Vi Sp 
Dy; = =a e = O(n?) 


where h = At is the assumed time interval. Substituting Eq. (5.44) in 
Eq. (5.43), we obtain a recursion relation for y; , ;, as follows: 


1 
MLSS gee 


[4 - 2h?) y,; + (h - Dy; _ 4] (5.45) 


This equation gives the value of y;,, provided that the two previous 
values of y are available, that is, y; and y;_,. To start the problem, 
two values of y will be needed. These will be obtained from initial 
conditions. Using central differences for the initial condition Dy, = 0, 
we have 


Dyy = ——— = 9 y4= 91 (5.46) 


Note that y_, in the above relation is inserted to make the derivative at 
Yo Symmetrical. It actually does not exist in calculations. In addition, 
we have the initial condition given in the problem yg = 1. Using this 
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condition together with Eq. (5.46) and setting i = 0 in Eq. (5.45), we 
can solve for y,. Having the values of yg and yy Eq. (5.45) will give the 
value of y,(i = 1). Thus, the solution can be continued for i= 3,4,.. ., 
resulting in values of y at discrete time steps n At where n = 0,1,2,3,... . 

Mathematically, one of the important problems in the numerical 
solution of differential equations is choosing the value h = At. If At is 
too small (in addition to the problems of the stability of solution which 
may arise), a great number of iterations will be required to obtain the 
value of y for a given time. For example, if the value of y at ¢ = 10 is 
required, a At = 0.01 will require 1,000 applications of Eq. (5.45), while 
a At = 0.1 willrequire only 100 applications of this equation for obtaining 
the desired value of y. On the other hand, if At is too large since the 
error is of the order of At? (or h? in Eq. (5.44)), the error in each 
application of Eq. (5.45) may accumulate, resulting in large errors in the 
solution. Some theoretical background will be given for calculating the 
effect of At on the size of accumulated error in Chap. 7. However, in 
most engineering applications, the value of At is chosen by numerical 
experiments. This means that a value of At is chosen and the problem 
run on the computer and solutions obtained. The value of At is then 
varied and the solutions examined. In this way, a value is chosen that 
is a tradeoff between computation time and the desired accuracy of the 
solution. 


DERIVATIVE OF A TABULATED FUNCTION 


The derivatives of tabulated functions can be obtained with the aid of 
relations already derived. These derivatives are represented in terms of 
backward, forward, and central differences in Tables 5-2, 5-3, and 5-4, 
respectively. For example, the central difference approximation of 
order h? for the second derivative of a function is given in Table 5-4 as 


2p2 
BoD ye = Ye Oy eee 


Applying this formula to tabulated values of y = logx of Table 5-5 of 
page 210, we can obtain derivatives at various values of x. For x = 1.02, 
we have 

_ 0.0043214 — 2(0.0086002) + 0.0128372 


2 
Dito =1.09~ (0.01) (0.01) ies 


= -4.18 x 10-9 
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INTEGRAL OF A TABULATED FUNCTION 


Using the symbol for the differential operator D, we can define an 
integral operator as the inverse of D: 


D-! f(x) = Integral of f(x) (5.48) 
From this, 
a+b ar 
f fod. = M2) = Litas m- fa 
la D 
a 
= E = 15) 
D 


Similarly, for the integration interval between a and a + nh, we have 


E"-1 


f(a) (5.49) 


Since the integration interval in the above equation starts at a and 
increases to a + nh, it is reasonable to derive the integral in terms of 
forward differences. Using the relations 


& 
f] 
_ 
ae 
> 


iS) 
u 
rile 
eS. 
1 
w]e 
fe 
oo |B, 
i] 
Stee 


in Eq. (5.49) and dividing the numerator by the denominator, we have 


at nb 
id f(x) dx 
la 


(2n — 3) (ee hn 
= nsf re IN gre ena 1 a a YN |p 
2 12 24 
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For n = 1, neglecting terms involving A? and higher, we have 


ath ne 
f f(x) dx h jl + =| f(a) 
@a 2 


ifr _ fa +h) - = (5.51) 
2 


LATO CHAT N 
2 


fT 


"1 


which can be recognized as the trapezoidal rule of integration. For 
n = 2, the term involving A® in Eq. (5.50) drops out. Neglecting terms 
involving A4 and higher, we have 


a+2b A2 
iy: f(x) dx = i(2 + 2A 4+ Vi (5.52) 


This can be readily evaluated by using the relation A = E - 1. This 
results in 


at+2b 2 
Hg ovr = i ADV Nie: eM Vi 
a 


h 


= = + 4E + E) f(a) S23) 


" 


{ia + 4f(a + h) + fla + 2h)) 


which can be recognized as Simpson's one-third rule of integration. 
Similarly, setting n = 3 in Eq. (5.50) and transforming in E operator 
form, 


a+3b 7 
f f(x) de = 2hUf(a) + 3fla +h) + 3fla + 2h) + fla + 3h] 
(5.54) 


This equation is usually referred to as Simpson’s three-eighths ruie. 


Numerical Methods and Calculus of Finite Differences 203 


Integrals of tabulated functions can also be obtained in terms of 
backward and central differences. The central difference relations can 
be evaluated by writing the integral of the function /(x) ina symmetrical 
region 


h 
[1 Be ate) orn" fla + nh) — fla ~ nh) 
‘a-nb D 


een D (5.55) 


nm pon 
BE a) 
D 


" 


For n = 1, this relation, from Eqs. (5.29), (5.30), and (5.39), can 
be written 


ath V2 _ R-1/2)(BV2 1/2} 
f flees (E EK aa 
‘a-h D 
- 284 F(a) 
D (5.56) 
= 2h dp fla) 
3 5 
pine feds 
6 30 
which, upon dividing, results in 
tb a2 2 
if fi) dx = 2h(1+— + 2844 ..-)f@ (5.57) 
la-b 6 15 
Neglecting the terms of 54 and higher gives 
ath 52 
¥ f(x) dx = 2h(1 + —)pl@ 
la-b 6 (5.58) 


W 


*if(a ~ Wh) + 4f(a) + fla +B) 


which is Simpson’s one-third rule of integration in central difference 
form. 
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For n = 2, Eq. (5.55) results in 


a+2b 1_ p-lypl -1 
of f(x) dx = (GE ME +E) fla) 
‘a~2h D 
_ (2h 8y)(8? + 2)* 
oa 3 5 (5.59) 
Die ee, 
6 30 
. 2(2 452, 194 ) 
45 
Neglecting the terms of fourth and higher orders in 5, we get 
meek 2h 
f f(x) dx = —[4f(a — h) - 2f(a) + 4f(a + h)] (5.60) 
a~2h 3 


Note that all integral formulas should give correct answers for f(x) = C = 
constant. This is usually used as a check of the integral formulas. For 
example, Eq. (5.60) for f(x) = C results in 


at+2b 2h 
of Cdx = 4khC = =(4C - 2C + 4C) = 4hC (5.61) 
a-2b 3 


Integrals of functions can similarly be obtained in terms of backward 
differences. For this, consider integration over the interval a — nh toa: 


2 fo dx = £22!" _ fla) = fla = nbn 
‘a-nb D ocak D 
oR 
eg esl 
D 
_1-0-W 


(continued) 


*This expression is obtained by writing § - E2 - e712 and squaring 
8? = E+E !- 2. This gives E+E! = 8 
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aln— Wop nln ~ Din 2o3_ 


! 1 
= 3. BL f(a) 


2 3 
oe ) 
Wedows 


(5.62) 


Division of this expression will result in 


n3 
fe f(x) dx = in-v mvt.) fa 
. 6 


This expression can be evaluated for different values of n. 


INTEGRATION FORMULAS BY 
INTERPOLATING POLYNOMIALS 

The integration formulas given above and additional integration for- 
mulas for the numerical solution of differential equations can be obtained 
by using polynomial interpolations. The basic idea is that a polynomial 
with unknown coefficients is assumed to pass through the sets of given 
points. Using the values of these points, the coefficients of the poly- 
nomial can be evaluated. These coefficients are then used to obtain 
integration formulas. 

Consider Fig. 5-1 of page 185 with variables x and y = f(x). Assume 
that the straight line 


y = Ax+B (5.63) 
passes through (x;, y,) and (x; , 4, ¥;, )- For simplicity, let us set x;= 0 


or equivalently take x; as the starting point and set the points (0, y;) and 
(Ax, y; , ;) in Eq. (5.63). This results in 


y; = B 
¥j41 = A(Ax) + B 


(5.64) 


Solving these equations for A and B, we obtain 
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Vier i 


kaha a (5.65) 


Using these values, Eq. (5.63) becomes 


Vier i 
y = ——— x+y, (5.66) 
Ax 


The area under the curve from x = 0 to x = Ax is obtained by integration 
of Eq. (5.66). This gives 


Ax y; -y 2 
A= A yar = HAM (ME) vac = Bo, 1 +99 
0 Ax 2 2 


(5.67) 


which is the trapezoidal law of integration. In a similar fashion, the 
Simpson rule of integration can be obtained by assuming the second- 
degree polynomial 


y = Ax? + Bx+C (5.68) 


passing through the points (—Ax, y,_ ,), (0, y,), (Ax, y;, ). Substituting 
these values in Eq. (5.68) and solving for A, B, and C, we obtain 


Vie. — Wit H- View Viz 
ia Geet Sg . SE 0 


2(Ax)2 2Ax 


Using these values of A, B, and C in Eq. (5.68) and integrating, 


Area = ti ydx = acral + 4y, + 9;_)) (5.70) 


Ax 


which is Simpson’s one-third rule of integration. 
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Expressions (5.67) and (5.70) give areas under the curve between 
points which satisfy expressions (5.63) and (5.68). It is sometimes 
desirable, as will be seen in predictor-corrector methods of solving dif- 
ferential equations (Chap. 6), to extend the range of integration using 
the same polynomials. For example, using Eq. (5.68) with the coeffi- 
cients (5.69), the area under the curve of Fig. 5-1 from —2Ax to 2Ax 
becomes 


2Ax 4Ax 
- ydx = — (29; ,) — 9; + 2Wi-y) (5.71) 
2x 3 


This expression gives the area under the curve extending from (x; _ 5, ¥;_ 9) 


to (x; , 9, ¥;4 2) in terms of values of y extending from y;_, to y; , )- 
Differentiating Eq. (5.68) twice will result in 


——= 2A (5.72) 


Substituting the value of A from Eq. (5.69), we obtain 


dy Viv Vit Yin ah 
2 Ax? S193 


a 


which is the central difference expression for the second derivative of y 
at x;. 
Differentiating Eq. (5.68) once, we obtain 


WY _ 2x + B (5.74) 


dx 


The value of this derivative at x = x; = 0 is obtained by substituting B 
from Eq. (5.69) and x = 0. This results in 


dy Yi+1- Vi-1 
= 5.75 
dx 2Ax ( ) 


which is the central difference expression for the first derivative. 
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The integrals derived here, in addition to being used in obtaining the 
integrals of tabulated functions, are used in the numerical solution of 
differential equations, as will be discussed in Chap. 6. 


GREGORY-NEWTON INTERPOLATION FORMULA 


Consider the function f(x) tabulated from x =.a to x = b at incre- 
ments of Ax = h. It is desired to find the value of f(x) at x = a + nh 
where n can be a fraction. Using the relation 


f(a + nh) = E" f(a) (5.76) 


and the expression for E in terms of forward differences, we have 


E" f(a) = f(a + nh) = (1+ A)" f(a) 


xs [: ie ay + nae D 2 _ n(n - aie 2) 3 a |p 
3! 


(5.77) 


The above equation can also be derived in terms of backward differences 
Y. For this we start from point b and write 


E"f(b) = f(b — nh) 


= (1 - ¥)"f(o) (5.78) 
- [r-nv «Ata Dye sin De 2, |r 
2! 3! 


Equations (5.77) and (5.78) are referred to as Gregory-Newton inter- 
polation formulas. Using these relations the values of tabulated functions 
can be obtained at any point within the interval by obtaining the required 
forward and backward differences. 

Example. Consider the table of log,)x for values of x between 1.0 
and 1.05 at increments of Ax = h = 0.01 to be given. It is desired to 
obtain logx at x = 1.024 by forward interpolation formula (5.77). 

The necessary calculations are represented in Table 5-5. Since the 
value of logx at 1.024 is desired, let us set f(a) = 1.02. The value of n is 


Pe a O2Ar= C20 vi, 


0.010 
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Using this value of n together with proper values of Table 5-5 in 
Eq. (5.77), and neglecting terms of fourth order and higher, we have 


£(1.02 + 0.4 x 0.01) = 0.0086002 + 0.4(0.0042370) 


, 0-4(0-4 ~ D (_9 9000409) (5.79) 


, 0.4(0.4 - 1)(0.4 — 2) 
6 


(0.0000008) 


From this 


£(1.024) = 0.0103000 
Note that the first two terms on the right of the equal sign in Eq. (5.79) 
represent the familiar linear interpolation terms. 
STERLING'S INTERPOLATION FORMULA 


Just as Gregory-Newton interpolation formulas are based on forward 
and backward differences, the Sterling interpolation formula is based on 
central differences. For the derivation of this, consider the Taylor series 
expansion of the function f(a + nh) 


2 3 
fla + nh) = ( + nhD + = nD? + eer + to (5.80) 
2! ! 


where, as before, f(a + nh) is the value of tabulated function at a + nh, 
h is the increment of independent variable x, and n can be a fraction. 
Substituting the values of D in terms of central difference operator 6, 
from Eqs. (5.39) and (5.40) in Eq. (5.80), we have 


3 5 2 4 
Seo ryiyeda (IS (edie a ae -E@e_&,...) 
6 30 2! 2 


3 5 5.81 
Bae 25.) .]re nee 
4 


Combining powers of 6 results in 
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Table 5-5. Example of Gregory-Newton Interpolation Formula 


x f(s) = logx Af (x) 
1.00 0.0000000 0.0043214 0.0000008 
1.01 0.0043214 0.0042788 0.0000009 
1.02 0.0086002 0.0042370 0.0000008 
1.03 0.0128372 0.0041961 
1.04 0.0170333 0.0041560 


0.0211893 


0.0000000 
0.0043214 0.0043001 
0.0086002 0.0042579 
0.0128372 0.0042165 
0.0170333 0.0041760 
0.0211893 


2 
f(a + nh) = [t+ma+ sats sea Oe 
2! 


2 2 
pA SS 2) os aS On |p 


Equation (5.82) is the Sterling interpolation formula. This equation can 
be used for finding the values of the function at any point within the 
intervals by obtaining the required differences. 

Example. As an example of application of this formula, let us again 
consider the problem of evaluating log 1.024 from tabulated values of 
logx as given in Table 5-5. The necessary values to be used in Eq. (5.82) 
are given in Table 5-6, The values of » 5, 5%, and » 5° are obtained from 
the following (previously given) equations: 


#8y; = 20;,1-9;-) 
(5.83) 
dy; = Viet or Pea 


3 2 
pd3y,; = (ndr(d2y,) 
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The value of n as obtained previously is 0.4. Using this value of n and 
the tabulated values of Table 5-6 and neglecting terms of fourth degree 
and higher in Eq. (5.82) results in 


#(1.024) = 0.0086002 + 0.4(0.0042579) + (—0.0000418) 


(0.4)? 
2 


(5.84) 
“ 2410-5 — (0.0000008) = 0.0103000 


where f(a) = 0.0086002 is used as the starting point. This result agrees 
to seven decimal places with the result from the previous example. 


PROBLEMS 


1. Derive the expression for Ady, as given by Eq. (5.8). 
. Derive the expression for dt y, as given by Eq. (5.11). 

3. Obtain relations 5.22f and g for differential operators D’and D4 in 
terms of backward difference operators. 

4. Obtain relations 5.27d and e for differential operators D3 and D4 in 
terms of forward difference operators. 

5. Obtain relations 5.40b and c for differential operators D? and D*in 
terms of central difference operators 6 and p. 

6. Suggest a method for solving the differential equation 


oe a) 
dt? 


with dy/dt = 1 and y = 2 at t= 0, using central differences of order 
of h?, 

7. Using the tabulated values of tan x for x = 0.50 to 0.60 radians in 
intervals of Ax = 0.02 


x tan x 
0.50 0.54630 
0.52 0.57256 
0.54 0.59943 
0.56 0.62695 
0.58 0.65517 


0.60 0.68414 


212 


10. 


11. 
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find the values of tanx at x = 0.55 radian using the Gregory-Newton 
forward interpolation formula and Sterling’s interpolation formula. 
Obtain the first and second derivatives of tan x at x = 0.54 using the 
above table and relations given in Table 5-4, for errors of the order 
h*, Compare these values to values obtained by differentiation of 
tan x. 

Using the tabulated values of tanx above, obtain the area under the 
tangent curve between x = 0.5 and x = 0.58 using trapezoidal and 
Simpson’s one-third (central difference formulation) rules. 

Take the equation 


y = Ax? + Bx +C 


and assume that it passes through the points (0, y;), (Ax, y; , ,) and 
(2Ax, y; , 9). Evaluate the coefficients A, B, and C and obtain 


2Ax 
Area = Hi y dx 
0 


Same as the above problem, but this time find 


3Ax 
Area = f y dx 
Ax 


12. Assuming the polynomial 
y = Ax3 + Bx? + Cx + D 
passing through (0, y,), (Ax, y;, ,), (2Ax, y;, 9), and (3Ax, ¥; , 3), 
find the values of A, B,C, and D. Show that the value of A is 
proportional to the third derivative of y with respect to x, d3 y/dx°. 
Compare the value of A with the forward difference Table 5-3. 
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6 NUMERICAL 
SOLUTION OF ORDINARY 
DIFFERENTIAL 
EQUATIONS 


The numerical solution of differential equations is accomplished by 
writing the differential equation in finite-difference form in terms of 
discrete time steps (At) and solving the resulting difference equation. 
This solution of the finite-difference equation will depend on the chosen 
value of time step At. In the limit when the value of At approaches 
zero, the solution of the finite-difference equation will approach the 
solution of the differential equation. It should also be noted that the 
analytic solution of a differential equation gives the value of the depend- 
ent variable x as a function of the independent variable t. This function 
can be evaluated for any value of the independent variable. On the 
other hand, the finite-difference solution of a particular differential 
equation will represent a set of values of the dependent variable x as a 


214 
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function of the independent variable t. The x’s will be obtained for 
values of independent variable 0, At, 2A, 3At, ..., and so on, It will be 
seen that the solution x, at n At, where n is an integer, will depend on the 
values of x for (n — 1)At, (n ~ 2)At, ..., and so on. This being the case, 
the starting values xp, x1, x5, ---, should be calculated in order to be 
used in the general expression of the finite-difference representation. 


METHODS OF STARTING THE SOLUTION 


Taylor’s Method 


This method of starting the solution of a differential equation consists 
of representing the solution in the neighborhood of the initial values in a 
Taylor series expansion. This expansion will be an exact representation 
of the solution in the neighborhood of the initial conditions. Having 
this expansion, the values of the independent variable x can be evaluated 
for the few starting values of At. 

Consider the first-order differential equation 


v= HL ftz,t) (6.1) 


dt 


with the initial conditions x = xq at t = t). Assuming the solution to be 
of the form 


x= Blt (6.2) 


we can write Taylor series expansion of x about t = t, as follows: 
= (ot ye eer Ba rete fev (6.3 
X= XQ + SO Oke ar SE eee = tp)’ xg + + a3) 


where primes denote derivatives with respect to t. These derivatives are 
evaluated at t). The first derivative is given by x’ of Eq. (6.1), and the 
second derivative can be obtained by differentiating this with respect to 
t. Higher derivatives can be obtained in a similar fashion. That is, 
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x’ = f(x,t) = 8, (x,t) 
zo One schidty SChtdee s Cen) Ey 


eae = + x = By(t,x,x’) 
dt ot dt ax dt ot ox 
0g 0g 0g 

Peace a ay cea ee By (t,x, xx”) ASA 
ox ot Ox" 

MP eB ME, Na hig 2 oy Re ey, 


where g’s are functions of indicated variables. Starting with the first of 
Eq. (6.4), the values of x’, x”, x”... are expressed as functions of pre- 
ceding available values. For example, x” is a function of t, x, and x’, all 
of which are available from previous calculations. Note that in Eq. 
(6.3) the values of derivatives are evaluated at the initial point Xq and ty 
around which the Taylor series expansion is written. Having the values 
given by Eq. (6.4), the series of Eq. (6.3) which represents the solution 
of Eq. (6.1) in the neighborhood of the initial condition can be written. 

Example. It is desired to evaluate the value of the function x expressed 
by the differential equation 


dx _ By x (6.5) 


for t = 1.1, 1.2, 1.3, 1.4, and 1.5. The initial conditions are t = 1, x = 1, 
Taylor series expansion corresponding to Eq. (6.3) can be written 


x= 14+(-De,+Le-vegrte- vies... (66) 
2! 3! 


where the values of the derivatives x’, x*, ... are evaluated at ty = 1. The 
set of equations corresponding to Eq. (6.4) can be written 


xy = (3 + p= 4,2=1) =2 
xy = G44 2,7 = 8 
xf = (6t+ x"), 4 = 11 


(6.7) 
x¥ = (6+ x), = 17 
xv = (x) 4 =.17 


BO ee (x) = 17 
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Thus the Taylor series expansion corresponding to Eq. (6.6) becomes 


ne a 1 _ 42 te )3 1 14 
x=1+t DQ) + F(t 1) (3) + Bt 0) () + Fo 1)*(17) 


ee OC) +e tte DEM) 4... (6.8) 
120 720 


The values of x can be evaluated using the terms given in Eq. (6.8) for 
the required values of t. These values are listed in Table 6-1. 

The solution of the differential Eq. (6.5) can be obtained by the 
methods described in the Appendix. This solution is given: 


x = Aet - 18 — 312 - 6-6 (6.9) 


where A is the constant to be evaluated from the initial condition 
t = 1,x = 1. This resultsin A = 17/e, or 


Ree iTees i= 1 = 87) —6t, =.6 (6.10) 


The corresponding values of x for t= 1.1, 1.2,... are given in Table 6-1. 
Note that only the values of x obtained for ¢ = 1.5 differ in the fifth 
decimal place. 


Table 6-1. Required Values of x for the Example Problem 
of Eq. (6.5) 


From Taylor series From solution 


Eq. (6.8) Eq. (6.10) 
1,0 (initial 1.00000 1.00000 
value) 
1.1 1.22691 1.22691 
ie} 1.51585 1.51585 
13: 1.88060 1.88060 
14 2.33701 2.33701 


15 2.90323 2.90326 
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Taylor Method for Higher-order and Sets of 
First-order Differential Equations 


The above method can be directly applied to higher-order differential 
equations. Consider the nth-order differential equation 


n 
x? = ax f(t, x, x.x%) 0.2, xP 74) (6.11) 

dt” 
with the given initial conditions x), Xqr Xr ees XH lat 1). The Taylor 


series for the variable x of this equation around 1, can be written 


(t-t) t- ty 
Te ary 


otc 


X= X%q + (t - ty) xg + x 


The values of the derivatives to ne are already available from the 
initial conditions. Additional derivatives as required in the Taylor series 
expansion can be obtained by using the initial values and Eq. (6.11). 

In addition to this method, the nth-order differential equation (6.11) 
can be reduced to n first-order differential equations and Taylor’s method 
of starting the solution applied to the set of these first-order equations. 
The reduction to n first-order equations is accomplished by defining 
new variables 


x’ = 2,(t, x) 


21 = xX" = zo(t,x,x’) = zo(t,x,z2,) 


2 = X™ = 2 (t,x, 24,29) (6.12) 
Z-1 = x" = f(t, x, x4 "%...,x7 1) 
= f(t, x, 24) Zp)--+s2,-4) 
where z,, Z5,..., 2, _, are newly defined variables. Thus the nth-order 


differential equation (6.11) is transformed to n first-order differential 
equations (6.12). 

Taylor’s method of starting the solution of a set of simultaneous 
differential equations is similar to the previously given procedure. To 
illustrate the application of Taylor’s method to higher order and sets of 
simultaneous equations, consider the following examples. 
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Example. Consider the following second-order differential equation 


7 ek 
Ce es 
dt? 


meer ate =O (6.13) 


with the initial conditions t = 0, x = 0, x’ = 1. The Taylor series repres- 
entation of the solution is written 


me ATF : 
X= Xy + 1X + XG + — XQ + + (6.14) 


The values of derivatives are obtained from the initial conditions and 
Eq. (6.13) as follows. 


X%y = 0 

x = 1 

S° = HX xy = 0 

x me — xX’ xp = -l (6.15) 
Beer x = 10 

xY = =x" xy = 1 


Se ir eee (6.16) 


This solution can also be obtained by writing Eq. (6.13) as two first-order 
differential equations 


(6.17) 
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and writing Taylor series expansion for x and z. This solution, in addition 
to the values of the function x, will give its derivative z. Of course, once 
the function in the form of series (6.16) has been obtained, the derivative 
can be obtained merely by differentiation of Eq. (6.16). The series of 
Eq. (6.16) can be recognized as the series for sint. Thus, the values of x 
can be obtained for t = 0.05, 0.10, . . . using the first three terms of the 
series as given in Table 6-2. 

Differential equation (6.13) is recognized as the second-order differ- 
ential equation representing the harmonic motion. The solution of this 
equation from elementary differential equation or the Appendix is 


x = Asint + Bcost (6.18) 
Using the initial condition t = 0, x = 0, x’ = 1, this solution becomes 
x = sint (6.19) 


which is the solution represented by the series of Eq. (6.17). The values 
of x evaluated using Eq. (6.19) are also given in Table 6-2. 

Example. Consider Taylor series expansion of the solution of the 
simultaneous system of differential equations 


—= x74 

dt 

° (6.20) 
LE ae ee 

dt? 


Table 6-2, Required Values of x for the Example Problem 


of Eq. (6.13) 
a — 
m 
Three terms of ; 
t series Eq. (6.16) Solution Eq. (6.19) 
0 (initial 0.00000 0.00000 
condition) 
0.05 0.04998 0.04998 
0.10 0.09983 0.09983 
0.15 0.14944 0.14944 
0.20 0.19867 0.19867 
0.25 0.24740 0.24740 
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with the initial conditions t = 0,x = 1, y = 0,y’= 1. Note that the 
first equation of the set is nonlinear as x appears to the second power. 
Equations (6.20) can be written.as a set of three first-order differ- 
ential equations by introducing z as the additional variable. This 
results: 


(6.21) 


Z=x+y-t 


with the initial conditions t = 0,x = 1,y=0,z2=1. The solutions 
expressed in Taylor series expansion about ¢ = 0 can be written 


2 8 
xX = X, + xy + —X%y + —XQ + 
ORG) 5g “0 ay 0 
ee ae (6.22) 
Y = Yo + ty + Yo ete 
2 3 
2 = 2 + tz pe zee zy + 
came ea a ao™ At 58 


The values of the derivatives at t = 0 can be obtained by starting with 
Eqs. (6.21) as follows: 


(continued) 
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j= (20x)? + 2x], = 8 
Yo = (2°), -9 =l1 
0 


ae Sy), 2g = 4 


(6.23) 
me SCAR inet a Dace” 2xx), 9 = 34 
yi = (2M), = 4 
aes (x 4 ene are 
Using the above values, the series of Eq. (6.22) becomes 
x = irr sea hyrateny  cL T a 
2 3 12 
Di eS ad 
a ere ere (6.24) 
2 6 6 
2 
2 Sie ara ee 
3 


The initial values of x, y, and z can be evaluated from the above series. 


Euler Method of Starting the Solution 


The Euler method of starting the solution of a differential equation 
in effect uses the definition of the derivative as the ratio of two discrete 
intervals and proceeds to calculate values in the neighborhood of the 
initial starting point. This method, in many applications, is also used as a 
method for continuing the solutions. Consider the differential equation 


OF lx,9) (6.25) 
dx 


with the initial conditions x = xy and y = yg. This equation can be 
written using the definition of derivative 


Eyer Hime (6.26) 
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For small Ax’s 


dy 
Ay ~ Ax 
ie (6.27) 


~ f(x,y) Ax 


Thus, assuming a Ax and using the function f(x,y) from Eq. (6.25), the 
value of Ay can be calculated. The value of y for the next increment in x 
will be y + Ay. Although this method of computation is used for 
starting solutions, better results are obtained by using Taylor’s method. 
Note that the increment Ax in Euler’s method can be made small to 
improve the accuracy of the solution. 

Example. Consider the differential equation 


dy x 


dx 1+y 


(6.28) 


with the initial conditions x = 0, y = 1. It is desired to find the values 
of y for x = 0.1 and 0.2 by Euler’s method. Let us take a Ax increment 
of 0.02 and write Eq. (6.28) in notation (6.27) 


2 
Ay = Ee) (0.02) 
+¥/ea0 
x20 (6.29) 


-0.01 


" 


Thus the value of y for x = 0.02 will be y = 1 — 0.01 = 0.99. We continue 
the solution by writing Eq. (6.29) for x = 0.02 and y = 0.99 as 


2 
Ay = ( = *) (0.02) 
1+ ¥,/~=0.02 


(6.30) 
= — (9:2898) (0:09) 
1.99 


—0,0099457 
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and the value of y becomes 
y = 0.99 — 0.0099457 = 0.980054 


Continuing the above operation for x = 0.04, 0.06, . .. we obtain values 
of y given in Table 6-3. From this table the values of y for x = 0.1 and 
0.2 can be obtained as required in the problem. Note that by assuming 
Ax = 0.1, Eq. (6.29) can be solved for Ay and, subsequently, y. This 
results in 


Bs. 
Ay = (: 7 (0.1) = -0.05 (6.31) 
l+y at 
y=l 
and 
y = 1.0 = 0.05 = 0.95 (6.32) 


This value differs from the more accurate value of y for x = 0.1 as 
given in Table 6-3. Continuing the operation with these values and the 
equation 


xy 
Ay*= G - (0.1) (6.33) 


0.1 
0.95 


Table 6-3. Values of y asa 
Function of x (Eq. (6.28)) 


x BA 
0.00 1.000000 
0.02 0.990000 
0.04 0.980054 
0.06 0.970171 
0.08 0.960359 
0.10 0.950627 
0.12 0.940982 
0.14 0.931435 
0.16 0.921993 
0.18 0.912665 


0.20 0.903461 
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the value of y at x = 0.2 becomes 
y = 0.95 + Ay = 0.901795 


which again differs from the corresponding value given in Table 6-3. 

Note that if the differential equation contains fractions of the variables 
as in Eq. (6.28), the application of Euber’s method for starting the 
solution is simpler than Taylor’s expansion. The reason for this is that 
Taylor’s expansion requires first- and higher-order derivatives which are 
tedious to obtain in case of fractions. 


METHODS OF CONTINUING SOLUTION OF 
DIFFERENTIAL EQUATIONS 


Having obtained several starting values of the dependent variable, we 
can use the general finite difference expressions for continuing the solu- 
tion. The values of time or space increments, Atand Ax, can be changed 
during the methods of continuing the solution. For example, if during 
the initial transients, a higher solution accuracy is required, a smaller 
increment can be used and the increment increased during the latter 
portions of the solution. In this chapter, we will discuss the methods of 
continuing the solution for initial value problems while in the next 
chapter the method of solutions of boundary value problems will be 
discussed. 

The initial value problems are the ones in which the values of the 
dependent variable and its derivatives are specified at an initial point of 
the independent variable (these problems are also referred to as one-point 
boundary value problems). The initial point refers to the start of the 
problem. For example, in the differential equation 


d?x 
dt? 


+x=0 (6.34) 


the initial conditions can be expressed as the values x and dx/dt at some 
value of t = t). The solution of the differential equation will then apply 
for values of t > ty. Since Eq. (6.34) is a second-order differential 
equation, two conditions will be required to specify the constants of the 
solution. In the above case, the values of x and its derivative constituted 
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these conditions. This is to be compared to boundary value problems 
where the conditions at various values of the independent variable are 
given. In the case of Eq. (6.34) if the values of x at tjand x at t, > to 
were specified, again the constants of the solution of the differential 
equation can be evaluated. Specifying these values converts the problem 
to a boundary value problem (also referred to as two-point boundary 
value problems), 

Most of the problems in electrical engineering are initial value problems 
as the independent variable of the differential equation is time. In 
these problems, conditions at time equal to zero are given and the 
solution is required for ¢ > 0. However, in mechanical engineering, 
boundary value problems are encountered in elasticity and heat transfer. 
Examples of these will be bending problems in strength of material 
where the conditions at the end points are specified. 

In this chapter we will discuss the numerical solution of initial value 
problems, and in the next chapter we will extend these techniques to 
boundary value problems. 


Milne’s Method of Solution of Initial 
Value Problems 


Consider the differential equation 


Be eer fu) (6.35) 


dx 
and assume that the first four points Yo: Yj» Yq ¥g3 Corresponding 
to Xx, X1,%y,xg are calculated by the previously given methods 
of starting the solution. Denoting the interval between x’s as 


Ax = x1 — Xg = Xq - X1 = Xg - XQ = --+ Eq. (5.71) of Chapter 5 can be 
written 
24x 
if lei ae ~ fy + 2f;-)) (6,36) 
—2Ax 3 


Note that in the above expression y’ = f(x,y) is substituted for y of 
Eq. (5.71). The left side of Eq. (6.36) can be written 


2Ax Vita 
af Y ae = f dy ae Sy (6.37) 
—~2Ax dx 
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where i is used as an index. Using Eq. (6.37) in Eq. (6.36), we get 


42> 22 = TE Oleg - fit Hiv (6.38) 


It is more desirable to write the expression for y, , , in terms of previous 
y’s. This can be obtained by merely setting the index i = i — 1 in the 
above expression: 


vee = Yat OF, ~ fant Mia) (6.39) 


Having the values of x, through x, and the corresponding values of yo 
through y,, the values of fy through /, can be calculated by substitution 
in f(x,y) of Eq. (6.35). Then by setting i = 3, 4, 5, 6,... in Eq. (6.39), 
the values of y4, ¥5,- + -for x4 = x3 + Ax, xg = x4 + Ax, ...can be calcu- 
lated. Equation (6.39) is referred to as Milne’s predictor formula. 

In some problems it may be desirable to check or correct the solution 
by another integration formula. This is accomplished as follows. From 
Eq. (6.39) we obtain the value of y,;, , corresponding to x; , ;- Setting 
these values in Eq. (6.35) will give f; , ;. Equation (5.70) of Chapter 5 
can be written in the form 


Ax ie 
i y’dx = Fe eee 
“Ax 


where y is replaced by y’. Replacing y’ on the right of the above equation 
by f, we get 


Vie = Oeae Seteaie Hye tran! (6.40) 


Equation (6.40) is termed the corrector equation. The value of y;, , 
obtained from Eq. (6.40) is substituted in Eq. (6.35) and a new f;, , is 
obtained. This value is then substituted in Eq. (6.40) and a second 
iteration of y;,, is calculated. If this y;, , and the previous y, , , agree 
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to some specified accuracy, then y; , 9 is calculated from Eq. (6.39); if 
not, the iteration given by Eq. (6.40) is repeated. However, in practice, 
a single application of corrector Eq. (6.40) is sufficient. It should be 
pointed out that in some applications the predictor-corrector method 
described above gives unstable solutions (see [2]). In addition, as seen 
from Eq. (6.39), Milne’s method requires four initial values computed 
by the previously given methods for starting the solution. 
Example. Consider the previously given differential equation (6.5) 


Cs ERs Fie (6.41) 


with the initial condition t= 1,x = 1. Assuming that the first four 
values of x as a function ¢ are computed (Taylor series solution of 
Table 6-1), the value of f(x, t) in Eq. (6.41) can be obtained as follows: 


ty = 1.0 xq = 1.00000 flag, to) = fy = 2.00000 
ty = 11 x, = 1.22691 f, = 2.55791 
ty = 12 xy = 1.51585 fo, = 3.24385 
tj = 13 x3 = 1.88060 fy = 4.07760 


Having these values, we can apply the predictor Eq. (6.39) with i= 3 and 
calculate x, (same as y4 of Eq. (6.39)) 


X4 = ty + a ef, = fy + fy) (6.42) 


Assuming that At = 0.1, this becomes 


X4, = 10+ Se x 4.07760 — 3.24385 + 2 x 2.55791) 
3 


2.33696 (6.43) 


ft 


which is the predicted value of x for t,=tg+ At=13+0.1= 14. 
Setting this value of x4 in Eq. (6.41), we get 


fa (% qs ty) = (1.4)3 + 2.33696 = 5.08096 (6.44) 
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Using this value of f, in the corrector equation (6.40) with i= 3, we get 


me ic ae + fs + fq) = 1.51585 + 9 (8.24885 +4 


x 4.07760 + 5.08096) = 2.33702 (6.45) 


which is the corrected value of x,. Assuming that a single application of 
the corrector formula is sufficient (x, in Eq. (6.45) is close to x4 in 
Eq. (6.43)), we calculate fa with the corrected value of x4 


fy = (1.4)3 + 2.33702 = 5.08102 (6.46) 


Now we return to the predictor equation (6.39) with i= 4 and calculate 
the value of x;. From this point the process is repeated, resulting in the 
values given in Table 6-4. In this table time represents the values of t, 


Table 6-4. Solution of the Example Problem Eq. (6.41) 


Shae PRED X CORR X FUNCTION | ANALT SOL 
(Eq. (6.43)) | (Bq.(6.45)) | (Eq.(6.46)) | (Eq. (6.10) 

1.0 1 1 2 1 

Ll 1.22691 1.22691 2.55791 1.22691 
12 1.51585 1.51585 3.24385 1.51585 
13 1.88060 1.88060 4.07760 1.88060 
14 2.33696 2.33702 5.08102 2.33702 
1.5 2.90320 2.90326 6.27826 2.90326 
1.6 3.59994 3.60002 7.69602 3,60002 
17 4.45071 4.45080 9.36380 4.45080 
18 5.48210 5.48220 11.3142 5.48220 
19 6.72415 6.72426 13.5833 6.72425 
2.0 8.21068 8.21080 16.2108 8.21079 
2.1 9.97970 9.97983 19.2408 9.97982 
2.2 12.0739 12.0740 22.7220 12.0740 
23 14.5409 14.5411 26.7081 14.5410 
2.4 17.4342 17.4344 31.2584 17.4344 
2.5 20.8135 20.8137 36.4387 20.8137 
2.6 24.7454 24.7456 42.3216 24.7456 
27 29.3039 29.3041 48.9871 29.3041 
2.8 34.5718 34.5720 56.5240 34.5720 
2.9 40.6409 40.6412 65.0302 40.6412 
3.0 47.6137 47.6140 74.6140 47.6140 


ay 55.6036 55.6039 85.3949 55.6039 
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PRED X refers to values obtained from Eq. (6.42), CORR X refers to 
values obtained from Eq. (6.45), and FUNCTION refers to values 
obtained from Eq. (6.46). The analytic solution which is the value of 
the solution given by Eq. (6.10) is listed in the last column of Table 6-4. 
As seen from the table the agreement between the analytical and finite 
difference solutions is excellent. 


Digital Computer Program for the Above Example 


The following digital computer program has been written in the 
BASIC computer language for a General Electric time-sharing computer. 
Although most of the statements of the program are self-explanatory, 
for further detail about this computer language the reader is referred 
to [3]. 


4 DIM T(100),X(100),2Z(100),F(100) 

5 DEF FNF(T)=17*EXP(T-1)-Tt3-3"(Tt2)-6"T-6 

10 PRINT “TIME,"“PRED X,” “CORR X,” “FUNCTION,” “ANALT SOL” 
11 PRINT 

20 LET T(0)=1. 

30 LET T(1)=1.1 

40 LET T(2)=1.2 

60 LET T(3)=1.3 

60 LET X(0)=1. 

70 LET X(1)=1.22691 

80 LET X(2)=1.51585 

90 LET X(3)=1.88060 

95 FOR I=0TO3 

100 LET F(I)=(T(I))t3+X(1) 

105 LET Z(I)=X(I) 

106 PRINT T(1),X(1),2Z(1),F (1), FNF(T(I)) 

110 NEXT! 

112 FOR I=3 TO 20 

115 LET T(I+1)=T(1)+.1 

120 LET X(I+1)=X(I-3)+((4".1)/3)*(2*F(1)-F(I-1)+2*F(1-2)) 
130 LET F(l+1)=(T(I+1)) t3+X(I+1) 

140 LET Z(1+1)=Z(I-1)+(0.1/3)*(F(1-1)+4*F(I)+F(1+1)) 
145 LET F(I+1)=(T(1+1)) t3+Z(1+1) 

150 PRINT T(1+1),X(1+1),Z(1+1),F(1+1),FNF(T(1+1)) 
155 LET X(I+1)=Z(I+1) 

160 NEXT! 

170 END 


The program starts with the dimension statement 4, reserving computer 
memory locations for time T predicted x, X, corrected x, Z, and the value 
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of the function f(x,i,F. Statement 5 defines the function FNF (T), 
which represents the analytic solution of the problem given by Eq. (6.10). 
Statements 20 through 90 enter the four initial conditions. Statements 
95 through 110 calculate the values of the function for these initial 
values, and in addition set the corrected values of x and the predicted 
values of x for the four initial points equal, i.e., statement 105. State- 
ments 112 through 160 calculate the answers. Statement 120 is the 
predictor equation (6.39) and statement 140 is the corrector equation 
(6.40). Note that after values are printed out by statement 150 the 
values of corrected x’s are substituted for predicted values—statement 
155. 

The program computes values for 20 time steps (statement 112), as 
given in Table 6-4. 


Stability of Solutions 


In numerical computations, problems are formulated in terms of find- 
ing the values of dependent variable y;, in terms of previously computed 
values of this variable, its derivatives, the independent variable, and the 
increment h or At. Denoting the available values of variables by x, 
through x,,, the problem can be written in equation form 


cal Crncm omega (6.47) 
Since the previously computed values of x’s contain errors that may 
have been caused by, for example, truncation of significant figures, or 
the use of a particular difference scheme, the error will be reflected in 
succeeding values of y’s approximately according to the relation 


Hie 
By) Sune (6.48) 


i i 
i={ Ox. 
j= j 


where dy, and 6x; are errors in the computed values of y; and x;. If the 
coefficients of 5x, terms, df,/dx;, are reasonably small, then 6x; errors 
will not degrade the solutions obtained for y;. In this case the solutions 
are termed stable. Note that these coefficients are also a function of the 
increment h. On the other hand, if the coefficients 5f,/5x; are large, then 
seemingly small errors in 6x; values are magnified by these coefficients 
and repeated iterations will increase the error in y,; values, rendering 
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the solution meaningless. Under these conditions the solutions are termed 
unstable. Note that Eq. (6.47) represents the particular finite-difference 
formulation of a problem and that the terms of Eq. (6.48) are a function 
of this representation. For this reason in a number of cases the actual 
solution of the problem may be stable but its finite-difference representa- 
tion may result in an unstable solution. Analytical methods of deter- 
mining the stability of a given finite difference representation will be 
discussed in the next chapter. 

Since the digital computers presently in operation carry a great number 
of significant figures, the computational errors and thereby the stability 
problems resulting from computations are usually small. In many engi- 
neering problems the inherent stability of finite-difference formulations 
is discovered when the problems are run on the computers and the 
solutions show behavior incompatible with physical and engineering 
reasoning. This method of analysis, of course, has no theoretical justifi- 
cation, but it reveals possible stability problems with the least amount 
of effort; theoretical analysis of stability criteria is extremely time 
consuming. Furthermore, in engineering applications the solutions are 
desired primarily during the initial transients, and most solutions that 
become unstable as the values of independent variable increases may be 
stable in the region of interest. 

To illustrate the above discussion we will analyze the errors in the 
finite difference formulation of the following example problem. 

Example. Consider the solution of the previously given example of 
Eq. (6.41). It is seen from Table 6-4 that the evaluated solution gives 
perfectly acceptable answers for the range of values of 1 < T < 3.1. Table 
6-4a gives the values evaluated to T = 10.0 under the identical conditions 
of time step and initial values. From this table it is noted that as T 
assumes larger values (although the solution remains acceptable), the 
error in the solution increases. The error is defined as the difference 
between the value given by the corrector formula and the analytical 
values obtained. 

From Table 6-4a it is seen that the percentage error (e.g., at T= 10.0 
the percent error is (1.75/136386) x 100 = 1.3 x 10%) remains very 
small. As discussed above, the error is partially caused by truncation of 
significant figures in the four initial values given following Eq. (6.41). 

As mentioned before, the stability of solution of predictor corrector 
methods has been studied extensively, as given in [2, 4]. In addition, 
[4] suggests a corrector formula developed by R. W. Hamming to replace 
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Table 6-4a. Error in the Computed Values Using Predictor-Corrector Method 


Value of solution 


Absolute value of 


Time T Corrector formula Analytic Solution error* 

1.0 1.0 1.0 0 

2.0 8.21084 8.21079 4.46 x 10° 
3.0 47.6143 47.6140 3.03 x 104 
4.0 199.455 199.454 1.32 = 10° 
5.0 692.173 692.168 4.94 x 103 
6.0 2157.04 2157.02 1.71 x 10? 
7.0 6320.34 6320.29 5.68 x 10° 
8.0 17884.9 17884.8 1.82 x 107 
9.0 49644.8 49644.3 5.69 x 107 

136388.0 136386.0 1.75 


*The values of errors were computed in the computer, which carries more significant 
figures than shown in the printout of the answers. 


Eq. (6.40). This formula in the previously given notation is as follows 


aon A [9y; - ¥j-2 + SAU, 1 + 2; - fy-2] (6.49) 


In [4] the errors developed using this equation and Eq. (6.40) are 
analyzed in the case of a particular example. It is shown that for this 
example the error in the solution using Eq. (6.49) as a corrector is less 
than using Eq. (6.40), which is Milne’s corrector equation. 


Adams’ Method 


Adams’ method of solving differential equations starts with the expres- 
sion for backward difference of a function in terms of differentials of 
the function, as given by Eq. (5.22a) of Chapter 5 


2 p2 3 p3 
Miss = (1-82 ER EP ve) aD, (6.50) 


where h is the increment of the independent variable. Substituting the 
values of differential operators D in terms of backward difference opera- 
tors (Eqs. (5.22)), we get 
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Vo. boo. 898 
Yu. = years Sy pee + +++) Dy; (6.51) 


In obtaining the above expression the relation 


Ue aiid ah hema 


is used. Note that in a given differential equation 


dy _ f(x,y) (6.52) 
dx. 


the value of Dy, = dy,/dx = f;. Using this, Eq. (6.51) becomes 


east reat (Ue oak Ato VS ee fy (6.53) 
yw 8 : 


This relation is Adams’ recursion equation for the solution of differential 
equations. Note that the values of y are based on the backward differ- 
ences of f;. 

In using the above relations let us assume that four initial values of 
y for xg through x, and the corresponding values of f, are calculated. 
That is, we have 


% % fo 
ia ale wlaate ta 
% Yo fo Vig V*fy 


x3 93 fs Vig V?fz Vf, 


where the differences (V’s) of f, are obtained from the given values. 
Having these values, y4 can be computed by setting i = 3 in Eq. (6.53) 


Vig. 5 3 
Wie 1 fy ae Fee (6.54) 
Aa Bs, (: iy 2 2 Dp fg + 8 fs 
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After obtaining y, another row in the above table can be computed and 
iteration can proceed to y,, yg, and so on. Adams’ method also requires 
several starting values to continue the computations. 


Fox-Euler Method 


This method can be considered an extension of Euler’s method of 
starting the solution of differential equations as described previously and 
depicted in Eq. (6.27). This equation can also be obtained from the 
Taylor series expansion 


pepines eas 
Ty ets lie Salle ed ara seca (6.55) 


by neglecting terms involving second-order and higher derivatives. This 
results in 


; d: 
Vint n= BY sas ORLY, 


which is identical to Eq. (6.27) with Ax = h. Suppose, in expression 
(6.55), instead of neglecting the second-order derivative term we sub- 
stitute this term with the value of y? in terms of forward differences of 
y’, that is, 
pS ES aah (6.57) 
h 


This will result: 


inh oye 
Ynys thy ho (6.58) 


Or, finally, we get 


y= EO +) (6.59) 
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which is the Fox-Euler integration formula. Note that the second term 

on the right of the above equation is the average value of the derivative 

between y;,, and y; multiplied by h. Equation (6.59) can be used 

without the application of Taylor series for obtaining the starting values. 
Example. Consider the solution of the differential equation 


BY eg 2in yoy’ (6.60) 


with the initial conditions x = 0, y = 1. The analytical solution of this 
equation using the initial conditions is 


y = 8e% — x2 _ 2x - 2 (6.61) 


Using Eq. (6.60) in Eq. (6.59) results in 
Viel = y+ 3B + 91 +2? +9) (6.62) 


Solving this equation for y; , ;, we get 


pepeaecl es h (2 2 
41" Tap p+ 3 a ae vi)| (6.63) 


By setting the values of i = 0, 1, 2, 3,...and assuming y = 1 for xp = 0, 
the above solution can be evaluated for increments Ax = h of x. This 
solution is given in Table 6-5 for h = 0.5. 

For comparison, the analytical solution which is the evaluation of 
Eq. (6.61) is also shown. 

Using Eq. (6.56), which is Euler’s method for starting the solution, 
the values of y can also be evaluated as a function of x. This will give 


CPi eata ee tes (6.64) 


which for the case of the example becomes 


Pee = 9; + h(x? + 93) (6.65) 


CL Bie hex? 
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Table 6-5. Solution of Eq. (6.60) 


y 
(Eq. (6.61)) 
1 


y 
(Eq. (6.65)) 
1 


a: 
(Eq. (6.63)) 
1 


.05 1.05135 1.05 1.05131 
“il 1.10558 1.10263 1.10551 
15) 1.16311 1.15826 1.163 
pA 1.22436 1.21729 1.22421 
.25 1.28978 1.28016 1.28958 
cs 1.35983 1.34729 1.35958 
35: 1.43501 1.41916 1.4347 
4 1.51585 1.49624 1.51547 
45 1.60288 1.57905 1.60244 
es) 1.69668 1.66813 1.69616 
PP) 1.79786 1.76404 1.79726 
6 1.90704 1.86736 1.90636 
65 2.0249 1.97873 2.02412 
on 2.15214 2.09879 2.15126 
75 2.28949 2.22823 2.2885 
38 2.43774 2.36777 2.43662 
85 2.59768 2.51816 2.59644 
o) 2.77019 2.68019 2.76881 
95 2.95616 2.8547 2.95463 
3.15655 3.04256 3.15485 


The solution resulting from this equation is also shown for comparison 
in Table 6-5. From this table it is noted that the values of y obtained 
from Eq. (6.63) more closely approximate the theoretical values of Eq. 
(6.61). The simpler equation (6.65) will give more accurate results if 
the time step is further decreased to, for example, 0.025. 


Solution of Simultaneous First-order 
Differential Equations 


The governing principles of the above solutions can equally well be 
applied to series of simultaneous first-order differential equations. As 
discussed previously and depicted by Eqs. (6.12), higher-order differ- 
ential equations can be written as a set of first-order simultaneous 
differential equations. Thereby, the methods of solution of simultaneous 
first-order differential equations is useful in solving higher-order equa- 
tions. 
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Consider the set of first-order differential equations 


y= flx,y,2) 
Ho (6.66) 


z 


9 (x,y, 2) 


where the derivatives are with respect to x and the initial conditions are 
X = X%q,¥ = Yo, aNd z= 2. Application of Adams’ method through 
Eq. (6.53) will result in 


Vv 5 v2 
Vie. = yra(ie lady red 


ath(s te Sve ey, 
pi 


(6.67) 


i 


where h is the increment in the independent variable Xo. After obtaining 
the first few values of y and z for initial x’s by Taylor’s method of 
starting the solution, Eqs. (6.67) can be applied much in the same way 
as for a single first-order differential equation. 

Example. Consider the previously given example problem of Eq. 
(6.21): 


x = x24 = flx,t) 
ice ee (6.68) 


Z=xt+y-te= ylxy,t 


where primes indicate derivatives with respect to t. Assume that the 
following initial values are given 


£ = los Xq Yor 2g» fo» Bor Yo (6.69) 


Neglecting terms involving V and V? in Eqs. (6.67), these equations can 
be written for x, y, and z as follows: 


Xin. = Xi + Af; 
Vier = ¥; + he; fold, 
z; = 2,+hp 
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Using conditions given in Eq. (6.69), the above equations can be solved 
for i = 0, resulting in x,, y,,and z,. Having these values, /,, g),and y1 
can be calculated from Eq. (6.68). Then the application of Eq. (6.70) 
for i = 1 will result in the values of x, yj,and zy. The solution can then 
proceed as before. 

Note that neglecting terms involving V and Vy? in Eq. (6.67) reduced 
this formula to Euler’s relation (6.27). 


Milne’s Method for Second-order Equations 


In addition to the method of solution described above, in which a 
second-order differential equation can be reduced to two first-order 
differential equations and then solved, methods exist for the direct 
solution of second-order differential equations. The first method to 
be described is that of Milne. This method is similar to the previously 
described Milne’s method for first-order differential equations. 

Consider the second-order differential equation 


y” = f(x) (6.71) 


where primes indicate derivatives with respect to x, with the initial con- 
ditions x = x9, ¥ = yo, and y’ = yg. The values of 91, 94; Yq) 9s gr 3 
corresponding to x), %5, and x, can be obtained using Taylor series 
expansion described for starting the problem. Having these initial 
values, fy, f1, fg, and f, can be computed from Eq. (6.71). 

Using Eq. (5.71) of Chapter 5 for y” instead of y, we get 


2Ax 
f y’ dx = aaciee! Seen) (6.72) 


where the function of Eq. (6.71) is used in the right of this equation. 
Since 


2Ax 2x yy e yi+2 
f y’dx = f oY dx = f dy’ = ¥5,0'- %¢-9 (6:73) 
-2Ax —2Ax dx Yi-2 
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Eq. (6.72) can be written 


. ‘ 4A 
Yien = Ying += Obiar- e+ ev (6.74) 
Setting the value of index i = i — 1, Eq. (6.74) can be written in a more 
convenient form as ; 
; B 4Ax 
oa Pina To trea pee (6.75) 


The above equation gives the value of y;,, in terms of four previous 
values of y’ and f. Having obtained the value of y;, , from Eq. (6.75), 
Eq. (5.70) of Chapter 5, Simpson’s one-third rule, gives 


Viel 


Ax (,, ees 
= pn + 5 Mie + dys + ¥3_4) (6.76) 


At this point we have all of the required values at i + 1 step, that is, 
X;,1 =X; + Ax, y;, 1 from Eq. (6.76) and y; , ; from Eq. (6.75). Having 
these values, /; , , can be computed from Eq. (6.71). 

At this point the corrector formula 


Vier = Fier t Sia + Mit hie (6.77) 


can be applied to obtain a better approximation for y¥;,, than the 
previous value as obtained from Eq. (6.75). Note that the primary 
difference between values of y;, , obtained from Eqs. (6.75) and (6.77) 
lies in the fact that in Eq. (6.75) the value of y; , ; is based on the values 
of f up to ith increment while in Eq. (6.77) the value of y;,, is based 
on the prior values of f as well as the current value, f;, ;. Therefore it 
is believed that Eq. (6.77) will give a better value for y;,,. Having 
this value of y;,,, Eq. (6.76) is reapplied and the value of y;,, com- 
puted. The procedure outlined following this equation may be reapplied 
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to compute another value of y;, , from Eq. (6.77), but in most cases a 
single application of the corrector formula (6.77) is sufficient. For the 
stability discussion of predictor-corrector methods the reader is referred 
to [4]. 


Noumerov’s Method of Solving Second-order 
Differential Equations 


When the first derivative term of a second-order differential equation 
is missing, Noumerov’s method of solution can be applied. Consider the 
following differential equation with derivatives with respect to x 


y” + f(dy = F(x) (6.78) 


with the »’ term absent. Substitution of y” in terms of its central differ- 
ence representation as given in Eq. (5.40a) of Chapter 5 will result in 


4 6 
(# ent oe « ~y tie ate (6.79) 


where A is increment in x. Multiplying both sides by (1 + 52/12), we get 


6 8 2 52 
(* + ae ae + )si + ht y; ro kre (f;y)) 
12 


240 = 15,120 
(6.80) 
2 52 
i PF; + ard F; 
12 


Expanding terms involving 52 in terms of their arguments, we get 


2 h 
incor) = tC eae ea ia Es wy fitvivd = het ged Nee 


n2 8& ~=—1388 
= h?F, + —(F, a7 + F,. -(2- Some er 
De eee eee AGRE Thao f 


(6.81) 


242 Digital Computer Methods in Engineering 


Neglecting the last expression in the above equation will result in an 
error of 0(h°). Solving the above for y; , ;, we get 


1 n? 5h? 
Vind = eters fia) + (0-%,, y; 
a eo os eis Seay 


(6.82) 


h? 
igh oa + 10F; + ria] 


which is Noumeroy’s recursive expression for the solution of Eq. (6.78). 
Example. As an example of applying Noumerov’s method, consider 
the problem of solving the differential equation 


Be Lea (6.83) 


with the initial conditions ¢ = 0, x = 1.0, and dy/dx = 0. Assume that 
the first few points of the solution are evaluated by Taylor’s method of 
starting the solution, as follows: 


t= 0 see 10 

t= 0.05 x = 0.99875 

t= 0.10 x = 0.99500 (6.84) 
0.15 x = 0.98877 

t= 0.20 x = 0.98007 

0.25 x = 0.96891 


Comparing Eqs. (6.83) and (6.78), we get 
f=1 and F=0 (6.85) 


Using these relations in Eq. (6.82), we get 


2 2 
i ateeet -( me Yin -(® és 2) (6.86) 
1 + (h?/12) 12 6 
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which is the recursion equation obtained from Noumerov’s method. 
Having two values of x, that is, x9 and x, corresponding to / and /,, the 
value of x5 can be obtained from the above expression. 

Now again consider Eq. (6.83), but this time let us substitute the 
central difference representation for the second derivative term as given 
in Table 5.4 of Chapter 5 


: — 2x. : 
Med are te 1 


+x, =0 (6.87) 
re ; 
From this we get 
X44 = X(2 - h?) - X04 (6.88) 


Note that the error in the above expression, from Table 5.4, is the order of 
h2 while the corresponding error Eq. (6.86) is in the order of h®. The 
recursion Eq. (6.88) also gives the value of x in terms of two previous 
values. Solutions given by Eqs. (6.86) and (6.88) are evaluated for At 
values of 0.1 and 0.20 and are given in Tables 6-6 and 6-7. The last 
columns of these tables give the values of the solution obtained from 


x = cost (6.89) 


which is the analytical solution of the problem. In Tables 6-6 and 6-7 
(E-2) represents 10. From these tables it is seen that Noumerov’s 
method of Eq. (6.86) is closer to the analytical solution for both h = 0.1 
and 0.2. And, in addition, the central difference solution of Eq. (6.88) 
improves as the time step is reduced from 0.2 to 0.1. For example, the 
central difference solutions from Tables 6-6 and 6-7, fort = 1, are 


h=02 t=10 x = 0.539189 
h=01 t= 1.0 x = 0.540302 


Analytical solution ¢ = 1.0 x = 0.540302 


Thus the solution approaches the analytical solution as h assumes smaller 
values. 
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Table 6-6. Solution of Eq. (6.83) for Time StepAt = h=0.10 


5 x 3 x 
Timet | a. (6.86) | (Eq. (6.88)) (Eq. (6.89) 
0 1 1 1 
L .995 995 -995004 
124 -980058 -98005 -980067 
3 .955324 .955299 .955336 
4 -921045 -920996 .921061 
5 .877563 .877483 .877583 
6 .825312 .825194 .825336 
7. .764815 .764654 .764842 
8 .696677 .696467 .696707 
0 .621577 .621316 .62161 
Ae .540267 .539951 540302 
1.1 453559 453187 453596 
i ye .362319 .361891 -362358 
13 .267459 .266976 .267499 
14 -169926 . 169391 -169967 
Des 7.06955 E-2 7.01128 E-2 7.07372 E-2 
1.6 —2.92413 E-2 —2.98669 E-2 —2.91995 E-2 
1.7 —.128886 —.129548 —.128844 
1.8 —.227243 —.227934 —.227202 
1.9 —.323329 —.32404 —.32329 
2: —.416185 —.416906 -.416147 


Table 6-7. Solution of Eq. (6.83) for Time StepA’ = h=0.20 


(Eq. (6.88)) 


0 1 
2 -980080 -980070 
4 921067 920937 
6 825345 824967 
8 696718 -695998 
1. 540315 -539189 
1.2 362371 360813 
1.4 169980 -168004 
1.6 —2.91869 E-2 | -3.15255 E-2 
1.8 —.227190 —.229794 
—.416137 —.418870 


b 
(Eq. (6.89) 
1 
.980067 
921061 
825336 
696707 
540302 
362358 
169967 
2.91995 E-2 
~.227202 
-.416147 
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The above solution can easily be programmed for a digital computer. 
The following is the program of the solution written in BASIC pro- 
gramming language. See [3]. 


DIM X(500),¥(500),Z(500) 
READ H,X(0),X(1) 

LET A=(((5/6)*H"H)-2)/(1+(1/12)*H*H) 
LET B=2-H°H 

LET Y(0)=X(0) 

LET 2(0)=X(0) 

LET Y(1)=X(1) 

LET Z(1)=COS(H) 

FOR I=1 TO 20 

LET X(I+1)=-X(I-1)-A*X(I) 
LET Y(I+1)=B*Y(I)-¥ (1-1) 
LET Z(I+1)=COS(H*(I+1)) 
NEXT I 

PRINT “FOR H="H 

PRINT 

PRINT “TIME”,“NOUMEROV”,”CENTRAL DIFF”,“ANALYTIC SOL” 
140 FOR I=0TO 20 

150 PRINT H*!,X(I),¥(I),Z(1) 
160 NEXT! 

170 DATA.1,1.,.99500 

180 END 


geagssssassssyse 


In this program X is the solution obtained from Eq. (6.86) by 
Noumerov’s method, Y is the solution obtained from Eq. (6.88) by the 
central difference method, and Z is the analytical solution of Eq. (6.89). 
Statement 5 reserves memory storage for these variables. Statement 10 
reads the data of initial values. Statement 20 computes the coefficient 
of x; in Eq. (6.86). Statement 30 computes the coefficient of x; in Eq. 
(6.88). Statements 40 through 65 set the initial values of central differ- 
ence Y(0), Y(1), and analytical solution Z(0) and Z(1). Statements 68 
through 100 compute the Noumerov solution given by Eq. (6.86), the 
central difference solution given by Eq. (6.88) and the analytical solution 
given by Eq. (6.89). The rest of the statements print out the solutions. 
The computer output corresponding to this run is given in Table 6-6. 


Runge, Heun, and Kutta Integration Formulas 


These integration formulas are widely used in the solution of differ- 
ential equations with digital computers. Their primary advantage lies in 
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the fact that no special method is needed for calculating starting values as 
was the case with some of the methods presented. For the complete 
derivation of the formulas, the reader is referred to [5]. In this section 
we will give the general method of this derivation and will present several 
commonly used integration formulas. 

Taylor series expansion of the dependent variable y; in terms of the 
independent variable x; around the point x;, y; can be written 


2 3 
Vay = 9) + Any) 4 Oy 2, Ay (6.90) 
2! 


where Ax = x;,, ~ x, and the superscripts in parentheses specify the 
order of derivatives. Thus, having the values of x,, y, and the derivatives 
yi, y;?, y,, ... the values of y; , ; can be evaluated using Eq. (6.90). 
The value of x; , ; is, of course, 


“ag a Xt k (6.91) 


where h is the increment in x. For a given differential equation 


yD a 2) Fa) (6.92) 
dx 


the values of the derivatives can be obtained as follows: 


yD = f(x,y) 

poe a a = f+ hf (6.93) 
x dy dx 

(3) 


"i 


y heey elie sileet fgpitt Tints, 

In these expressions x and y subscripts represent partial derivatives with 
respect to x and y variables. Evaluating these derivatives at x,, y; and 
setting in Eq. (6.90), we get 


Yin = H+ ih + 3 (fe + fy (6.94) 


+ § [lex + Yayl + fyyl? + + FO] RB + -- 
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where subscripts i refer to values at x;,y;. This formula represents the 
third-order approximation to the values of y as the fourth-order terms 
(terms containing h*) are neglected. 

Equation (6.94) is the basic equation in developing Runge’s integration 
formulas. Note that using only the first two terms on the right of the 
equal sign of Eq. (6.94) reduces this equation to Euler’s method. In this 
case, the error will be of order h? instead of h* of Runge’s formula. 

Using the definitions 


A’y = f(x,y) Ax = fh 
Avy = f(x + mh, yo + m A’y)h (6.95) 
Avy = flx + Ah, yo + pA*y + (\- p) A’y]h 


where m, A, and p are constants, it is possible to expand the above 
functions in Taylor series and sum up the resulting equations in the 
form 


pe tian ~ 2 (6.96) 
aN’y+ bA*y +cA”y 


" 


where a, 6, and c are again constants. Equating the terms of Eq. (6.96) 
with those of Eq. (6.95), after a considerable amount of algebra, relations 
between the constants m, A, p, a, b, and c can be obtained. Runge’s 
formulas are obtained by making certain assumptions in using these 
relations. For the complete derivation the reader is referred to [5]. 
Here, however, we will give several commonly used integration formulas 
obtained by the above procedure. 
1. Heun’s formula: 


DOD = Vi41 - 2; 
oe (6.97) 
= qhy + 3A y); 
where 
A’y = filx,yh 
A’y = f, (x + thy + Lay)h 


co 
a 
" 


f(x + 2h.y + 2a°yh 
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2. Kutta’s third-order rule: 


(6.98) 


Ay, = 3 (Ay + 4A”y + Ay); 


where 

A’y = f(x, yh 

oe 1 a 
A’y = filet shyt Ayah 


A”y f(x + hy y + 2A”y - A’yh 


Note that this formula is analogous to Simpson’s one-third rule. 
3.  Kutta’s fourth-order approximation (Kutta-Simpson one-third 


rule): 


Ay, = ZiA’y + 2A”y + 2A“y + AYy); 


where 
A’y = f,(x,yh 

es 1 tee 
My = f(x+ bey + Lay)h ey 


A”y = f(x + ghey + LA’y)h 
AY’y = f(x +hy + A”yh 


4. Kutta-Simpson three-eighths rule: 
Ay; = a (A’y + 8A”y + 3A”y + AN’y) 


where 
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A’y = f,(x, yh 
Avy =f; i Ln’ 
y f,(e+ thy + Ld’y)h (6.100) 
A~y = 2 eye LN 
Avy = f(x+ 2h y+ A’y Lavy)h 
Avy = flx+h,y + A”y - A*y + A’yh 


The error in using Eqs. (6.99) and (6.100) is of the order h® as 
opposed to order h* of Eqs. (6.97) and (6.98). In the derivation of the 
fourth-order equations an additional term containing h‘ was utilized in 
Eq. (6.94). 

As mentioned previously, in using Runge’s formulas no special method 
is needed in obtaining the initial values as was the case of predictor- 
corrector methods. In many applications the requirement of using a 
special starting method is considered a disadvantage. For this reason and 
because of its accuracy, Runge’s method is widely used in digital com- 
puter solution of differential equations. 

Example. As an example of Runge’s method of solution, consider the 
differential equation 


-y» y(0) = 0.5 (6.101) 


f(x,y) = —=e 


The analytic solution of this equation is 
y = e* — 0,5e73* (6.102) 


The program can be formulated for numerical solution by using, for 
example, Kutta’s third-order rule of Eq. (6.98), This can be done by 
assuming a value for Ax = h and systematically evaluating A’y, A”y, A” y 
of Eqs. (6.98) at x9 = 0, 9 = 0.5. The resulting values are then used in 
the first of Eqs. (6.98) to obtain 


¥1 = % + Aro (6.103) 
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Similarly, for yj, ¥3, and so on. For comparison, solutions were also 
evaluated using only the first two terms of Eq. (6.94)—Euler’s method, 


Vier = + fh (6.104) 


The results of these evaluations are given in Table 6-8 for h values of 
0.01, 0.05, and 0.1. From this table it is seen that Kutta’s formula 
(6.98) gives answers correct to four significant figures with an h = 0.1, 
while Euler’s method can be considered acceptable at h = 0.01. 

In practice, the value of the increment A to be used is arrived at by 
experimenting with the solution on the computer. In the case of the 
above problem, the problem is run with h = 0.1 and h = 0.05; if the 
answers do not change appreciably in the region of interest, the higher 
value of h(0.1) is chosen. The mere fact that the solution changes when 
the time or space increment is changed points to the fact that inherent 
errors existing in the finite difference representation of the problem are 
magnified by the value of the increment. For example, consider the 
values of the solution by Euler’s method of Table 6-8 at ¢ = 1.0. It is 
noted that the solutions vary as h is decreased from 0.1 to 0.05 to 0.01. 
If this were the only method being used one might use h = 0.005 to see 
if the compounding of errors degrades the solution before deciding on 
the final value of h. However, in decreasing the values of h the reader 
should be aware of the problems of the stability of solution which 
might arise. 

Computer Program for the Evaluation of the Above Solution. The fol- 
lowing computer program in a version of FORTRAN programming 
language [6] was written to implement the above solution. In the 
program, Y2 refers to solution obtained by Eq. (6.98), Y3 by Eq. 
(6.104), and Y4 by Eq. (6.102). Statement 110 defines the function of 
Eq. (6.101). Statements 120 through 150 are constants. Statements 
170 through 220 are mechanization of Eqs. (6.98), while statement 
(230) represents Eq. (6.104). The analytical solution of Eq. (6.102) is 
given by statement 250. The results of the program are given in Table 
6-8. 


100 DIMENSION Y2(250),Y3(250),Y 4(250),X(250) 
110 FXY (X,Y)=EXP(-3.*X)-Y 

120 Y2(0)=.5 

130 Y3(0)=.5 
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Table 6-8. Kutta’s Third-order Rule in Solving Eq. (6.101) 


Analytic 


Increment 


Time t solution (6.102) Ax =h 
‘00 50 
10 5344 
20 5466 5443 
30 5402 5375 
40 5226 5197 
50 4950 ‘4978 4950 
60 4662 4689 4662 
70 4354 ‘4379 4354 
80 4040 4062 “4040 
‘90 3730 3750 3730 7 
1.00 3430 3447 3430 h=0.01 
1.10 3144 (3159 3144 
1.20 2875 (2888 2875 
1.30 2635 2624 
1.40 2400 2391 
1.50 2183 2176 
1.60 “1983 1978 
1.70 ‘1801 1796 
1.80 1634 ‘1630 
1.90 1481 1479 
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140 Y4(0)=.5 

150 H=.05 

160 -DO10I=0,199 

170 | Al=! 

180 | X(I)=AI*H 

190 | DIY=FXY(X(1),¥2(1))*H 

200 | D2Y=FXY(X(I)+H/2.,Y2(1)+D1Y/2.)*H 
210 | D3Y=FXY(X(I)+H,Y2(1)+2.*D2Y-D1Y)*H 
220 | Y2(1+1)=Y2(I)+(D1¥+4.*D2Y+D3Y)/6. 
230 | Y3(I+1)=Y3(I)+FXY(X(1),Y3(1))*H 

240 | X(I+1)=X(I)+H 

250 | Y4(1+1)=EXP(-X(I+1))-EXP(-3.*X(I+1))/2. 
260 “10 

270 PRINT,” TIME Y2 ¥3 Ya" 
280 PRINT 

290 -DO 20 1=0,200,10 

300 [Panvr.xeb.v2u.va0,van 

310 “20 

320 STOP 

330 END 


Runge-Kutta Formulas for a Set of 
Simultaneous Equations 


As discussed in the beginning of this chapter, higher-order differential 
equations can be reduced to sets of simultaneous equations and these 
equations solved by various methods. One method in common use is the 
Runge-Kutta method, which is in turn an extension of the given formulas 
for the solution of first order equations. Consider the simultaneous 
equations 


dy 


= f (x,y,z) 
dx 
(6.105) 
ee = g(x,y, z) 
dx 


For example, Kutta’s fourth-order approximation of Eq. (6.99) in this 
case becomes 


Ay; = p(A’y + 2A*y + 2A”y + AYy) 
(6.106) 
Az. = 4 (Az + 2A°z + 2A”z + Az) 


i 
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where (for Ax = h) 


A’y = f,(x,y,2)h 
N’z = @,(x,y,2z)h 
A’y = f(x + zh y+ pA’y, 2 + 3 Mz h 


A*z = g(x+dhy+idy2+ 4h h 

(6.107) 
a 1 AZ 1 A”, 
Avy =f,(x+ shyt gz + 5h 2)h 


~ Pans ane Te 
Avz = B(x+ahy+ Fh y2+ 5h zh 
AYy = f(x + hy + Avy, 2+ A”zvh 


AYz = B(x + hy y + A”y, z+ A” z)h 


If these increments in y and z are computed in the order given, only 
previously computed increments will be needed at each step in compu- 
tation. 


Solution by Direct Substitution 


In a great number of problems it is possible to obtain solutions by 
directly substituting difference equations in place of differentials of a 
given differential equation. This was done in the case of the example 
problem in Eq. (6.87). In many closed-loop servo problems this method 
of direct substitution is the most practical way of solving the problem. 
In applying the method, care should be exercised to choose a time step 
which will give satisfactory solution. In engineering applications, this 
time step is chosen by experiments. That is, solution is obtained for a 
certain value of the time step and then this value of the time step is 
halved to see if the solution changes. If the solution remains practically 
the same, the original value of the time step is doubled and a solution 
obtained. The idea behind this is to obtain the largest practical value of 
the time step without detracting from the accuracy of the solution. 
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To illustrate the method of direct substitution, let us consider the 
following closed-loop navigation problem. 

Example. In this example we will illustrate the digital computer 
simulation of the proportional navigation which is often employed in 
missile guidance [7]. To derive the appropriate differential equations of 
this navigation scheme, consider the two-dimensional geometry of Fig. 
6-1. In this figure a missile centered coordinate system is used. The 
missile-to-target line-of-sight vector is designated by R and the closing 


AT 


Missile target 
line of sight 


Missile 


figure 6-1. Two-dimensional diagram of proportional navigation. 


rate along this vector as R. The navigation scheme is based on the 
following relation 


Ay = Ap = Ap (6.108) 


which states that the acceleration of the missile perpendicular to R, Ay, 
should be equal to the acceleration of the target perpendicular to R, Ap, 
and equal to the rotational acceleration of line-of-sight R, Ap: From this 
figure it is noted that this will keep the line-of-sight R parallel and since 
R is the rate of decrease of the vector R, an intercept is assured. The 
line-of-sight acceleration Ap will consist of two parts, the acceleration 
perpendicular to R which will equal the rate of change of tangential 
velocity Rw where w is the angular rate, and the vectorial change of 
R(=R dé). This will result in 
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hy, SAC ray (6.109) 
dt dt 


Differentiating this expression and using the relation 

o= — (6.110) 
where @ is the angle of R with respect to some reference 

Ap = Ro + 2oR (6.111) 
Setting this value in Eq. (6.108) results in 

Ap - Ay = Ro + 2ok (6.112) 


Using differential operators D and D?, Eq. (6.112) becomes 


Ay - Ay = RD?6 + 2k D0 (6.113) 
or 
rere 
———— (6.114) 
R D2 + 22D 


This equation can be written in servo notation of [8]. The servo loop 
of Eq. (6.114) can be obtained by assuming that the missile acceleration 
Ay is proportional to the angular rate of @ or Dé. In addition, let us 
assume the current value of range R as given by the equation 


R = Ry + iR (6.115) 


where R, is the initial value of R at t= 0 and ¢ is the time. Using this 
relation in Eq. (6.114) we can draw the servo loop shown in Fig. 6-2. 
In this figure at point a we have the numerator of Eq. (6.114) and at 
point b we have the value of 0. Assuming that the commanded missile 
acceleration is proportional to Dé, we have 


Ay = k DO (6.116) 
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figure 6-2. Servo loop of proportional navigation 
problem. 


where k is the proportionality constant. Using Eq. (6.116) we get to 
point c of the loop of Fig. 6-2. From this point we get back to a and the 
loop is closed. Note that integrating acceleration A,, twice will give the 
position of the missile, P of Fig. 6-2. 

As a numerical example of simulating the loop of Fig. 6-2, consider 
the following values and Fig. 6-3. A missile with a velocity of 1,500 fps 


V2 = 1,500 ft/sec V, = 500 ft/sec 
Mt ee Target 


| 20,000 ft | 


figure 6-3. Numerical values of the proportional 
navigation problem. 


is fired at an approaching target with a velocity of 500 fps at a range of 
20,000 ft. At this instant the target maneuvers with a vertical accelera- 
tion of 32 fps, keeping its horizontal velocity constant. Assuming that 
the missile horizontal velocity remains constant, what are the missile 
vertical accelerations, using proportional navigation? To write the digital 
simulation of the problem, we use the central difference methods of 
Chap. 5. Let us denote 


ae Agnes Ul (6.117) 


Equation (6.114) can be written, using central differences for the deriva- 
tives as 


ee Ogee 20 Renae G 
(Gast) eS Gus, 
; (An)? 2At 
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This equation can be solved for 0; , , in terms of 6;and 0;_,. Having the 


value of 6;, 4, 


acceleration A,, can be obtained from (6.116): 


0 
(Ay), 12 k 


The position of the missile P can be obtained from 
D?P = Ay 


or, in finite difference form, 


P. 1 ~ 2P; + P;_1 


a) 


At? 


point b of the loop of Fig. 6-2, the value of missile 


(6.119) 


(6.120) 


(6.121) 


This equation can be solved for P; , , in terms of P,; and P,_,. Thus the 
loop of Fig. 6-2 can be started by setting initial values of 0, Ay, and P 
equal to zero and introducing target acceleration Ap. The positions of 
the missile and target with respect to the initial missile position can be 


obtained by using the following relation: 


Missile (horizontal) = Vot = 1,500¢t 


Target (horizontal) = Rg - V,t = 20,000 — 500¢ 


Missile (vertical) = P (Eq. (6.121)) 


Target (vertical) f Ap? = 16¢? 


(6.122) 


The following digital computer program has been written in BASIC 
programming language (see [3]) to accomplish the above procedure. 


In the program, the following notations are used: 


L = angle @ in radians 
A missile acceleration, ft/sec? 
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110 
120 
130 
140 
145 
150 
160 
170 
180 
190 
200 
205 
210 
220 
230 
235 
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P= missile vertical position, ft 

RO = Ry initial value of R, ft 

V1 = target horizontal velocity, ft/sec 

V2 = missile horizontal velocity, ft/sec 

K = proportionality constant of Eq. (6.116) (ft/sec? )/(rad/sec) 
Al = target vertical acceleration, ft/sec? 

T2 = time step size At, sec 


R1 = R orclosing rate = ~(V1 + V2) 


DIM L(500),A(500),P(500) 

READ RO,V1,V2,K,A1,T2 

LET R1=-(V1+V2) 

PRINT “RZERO”“RO,”TGT VEL"V1,"MSL VEL" V2 
PRINT “K"K,“TGT ACC” A1,"DEL T’T2 

PRINT 

LET L(0)=0 

LET L(1)=0 

LET A(0)=0 

LET A(1)=0 

LET P(0)=0 

LET P(1)=0 

LET 11=RO/(-R1*T2) 

FOR I=1 TO 11-2 

LET T=I*T2 

LET U=A1-A(1) 

LET R2=RO+T*R1 

LET L(I+1)=U*T2*T2+L (I) *(2*R2)+L(I-1)*(R1*T2-R2) 
LET L(I+1)=L(1+1)/(R2+R 1*T2) 

LET A(1+1)=((L(1+1)-L(1))/T2)*K 

LET P(I+1)=T2*T2*A(1)+2"P(I)-P(I-1) 

NEXT I 

LET P(11)=T2*T2*A(11-1)+2*P(11-1)-P(1 1-2) 

PRINT “TIME”,“TGT POS H”,“TGT POS V”, 
PRINT “MSL POS H”,""MSL POS V"” 

FOR J=0 TO 11 STEP 20 

PRINT J*T2,RO-J*T2*V 1,(A 1/2) *(J"*T2)12,J*T2*V2,P(J) 
NEXT J 

LET J=I1 

PRINT 

PRINT J*T2,RO-J*T2*V 1,(A1/2)*(J*T2)t2,J*T2*V2,P(J) 
PRINT 

PRINT “TIME’,“ANGLE”,“MSL VER ACC” 

FOR J=0 TO 11 STEP 20 
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240 PRINT J*T2,57.3*L(J),A(J) 

250 LET J=I1 

255 PRINT 

260 PRINT J*T2,57.3*L(J),A(J) 

270 DATA 20000,500, 1500,20000,32,.025 
280 END 


Statement 10 of the program reserves storage locations for L, A, and P. 
Statement 20 reads the data as given in Statement 270. These values are 


RO = 20,000 ft 

V1 = 500 ft/sec 

v2 = 1,500 ft/sec 

K = 20,000 (ft/sec? )/(rad/sec) 
Al = 32 ft/sec? 

T2 = 0.025 sec 


Statement 25 sets the value of rate of change of R equal to the sum of V1 
and V2 with a negative sign (since R decreases as the missile and target 
approach each other). It is assumed here that the angular change of 
vector R is small and R remains practically horizontal during the flight. 
Statements 30 through 36 are print statements which write out the 
input data. Statements 40 through 51 set initial conditions equal to zero. 
Statement 65 computes the upper bound of time index J, This value will 
be equal to the initial range divided by distance moved at each time step. 
Statements 70 through 140 are the main loop of the program which com- 
pute L(I+1) = 0,;,, from Eq. (6.118), A+ 1) = (A),.1 from Eq. 


(6.119), and Pil + 1) =P;,, from Eq. (6.121). Note that the last 
position of the missile P(/1) is computed in statement 145, since at the 
last point the missile-to-target angle L according to statement 110 will 
go to infinity as the denominator of L goes to zero. Statements 150 
through 190 print time, horizontal target position (TGT POS H), vertical 
target position (TGT POS V) and horizontal and vertical missile positions 
(MSL POS H and MSL POS V), respectively. Note that in print state- 
ment 180 Eqs. (6.122) are used. Since statement 170 is indexed every 
20 time steps there may be cases where the values at the last computed 
time step will not be printed. For this reason statement 210 prints the 
last computed values. 
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Statements 230 through 260 print out time, angle L = 6 (in degrees), 
and missile vertical acceleration A (MSL VER ACC). 

The output listing of the computer run is given in Table 6-9. (This 
output is obtained by combining the outputs of statements 170 through 
260.) A graph of the horizontal distance and vertical distance with 
time ticks is shown in Fig. 6-4. 


Nonlinearities and Random Inputs 


In digital computer programs nonlinearities describing the limits of 
variables can be entered through logical instructions. For example, if it 
is desired to limit the missile acceleration Ay of Fig. 6-2 at point c toa 
predetermined value of +k,, this can be entered as a logical operation at 
point c and correspondingly in the computer program. The following 
flow chart describes this. 


Random variables can be introduced equally well into digital computer 
programs. Almost all of the digital computer installations have built-in 
routines for the generation of random numbers y, with uniform 
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o 
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figure 6-4. Graphical representation of solution given in Table 6-9. 


distribution, between zero and one. For this distribution mean and 
standard deviations are given as 
Mean = p(y) = 3 
(6.123) 
Standard deviation = o7(y) = + 


Since many of the errors in engineering systems can be represented by 
normally distributed white noise,* it is desirable to obtain random 
numbers with this distribution. Using numbers y from a uniform distri- 
bution (0, 1), it is possible to obtain numbers v from a normal distribution 
N(u,o), where » is the mean and o is the standard derivation. To 
accomplish this the following procedures are used: 

1. Approximate method—From the central limit theorem [9] 
using Eqs. (6.123) we can obtain the following relations 


ite 62m (6.124) 


*The term white noise is used when errors are independent of each other and are not time 
correlated. 


Numerical Solution of Ordinary Differential Equations 263 


For most applications k > 6 produces a distribution of v closely approxi- 
mating a normal distribution N(y,0). From Eq. (6.124) k = 12 isa 
particularly desirable choice, since the division by the radical is eliminated 
and Eq. (6.124) reduces to the form 


12 
ae (Sader (6.125) 
i=1 


In this expression, y,, as before, represents a random number from a 
uniform distribution (0, 1). 

2. Exact method—Using statistical analysis (see [10]), it is possible 
to obtain two random numbers from a normal distribution N(0, 1) by 
using two random numbers from a uniform distribution (0, 1), Assuming 
that the random numbers are y, and yo, we have 


2, = V-2Iny, cos (2ry5) 


(6.126) 
2 = y-2Iny, sin(27yo) 


where z, and z, are two random numbers from a normal distribution 
N(O, 1). To transform these to two random numbers from a normal 
distribution with mean value of » and standard derivation 0, N(y, 0), we 
write 


vy) = zZ0+4 oe 


Ug = Zo + I 


where v, and v, are numbers from the normal distribution N (y, 0). 

Method 2 is more accurate but it uses more computer time as it 
requires the evaluation of logarithmic and trigonometric functions. The 
first method gives sufficiently accurate results and is commonly used. 

In the previous example of Fig. 6-2, white noise of N(y,o) can be 
introduced, for example, on the values of missile acceleration Ay by 
computing v values from the above equations and adding to the values of 
acceleration at each time step. The bias errors (if they exist) are treated 
as constants and are directly added to the values of the variables. 
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Aside from the problems discussed above, in many computer applica- 
tions it is desired to obtain estimates of the mean x and the standard 
deviation s of random parameters. These values are computed using the 
equations 


el 
" 


(6.128) 


where x, is the parameter or the variable and M is the number of samples 
available. The last equation for s is more suitable for digital computer 
calculations since the value of x is not needed for keeping the running 
sums of cls The need for the computation of the mean and standard 
deviation primarily comes in reduction of data and in the case of Monte 
Carlo programs. These are programs with random inputs which are run 
many times to obtain statistical distribution of the parameters of the 
solution. For example, introducing noise with a normal distribution, as 
discussed above, at point c of Fig. 6-2 will result in parameters such as P 
and @ as a function of time with noise superposed. If the statistical 
behavior of these parameters at certain instances of time is required, the 
program should be run many times (say, about 100)* and the mean and 
sigma values calculated at these required times. 


PROBLEMS 


1. Obtain Taylor series expansion of the solution of the equation 


*The number of times a Monte Carlo program is run (or equivalently the number 
of data points) depends on the desired accuracy and confidence intervals of statisti- 
cal parameters (see [9]). 
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dt 6 = 3x 


dt 


about the initial condition x = 0 at t = 0. From this series, obtain 
the value of x at t= 0.2. Compare with analytical solution. 
Obtain Taylor series solution of the simultaneous equations 


dz 
—=x+y 
dx 
dy 
—=X+2 
dx 


about the initial conditions x = 0,z=1,y = 1. 
Obtain the Taylor series expansion of the equation 


2 
ey = -8x2 4 Qy 


dx2 

about the initial conditions x = 1, y = 1, dy/dx = 0. From this 
series, obtain the values of y for x = 1.1, 1.2, and 1.3. 

Obtain the solution of the differential equation 


dy 4x7 + 9? 


dx 4y 


with the initial conditions y = 1 for x = 2, for x = 2.1, 2.2, and 2.3, 
using Euler’s method. 
Obtain the solution of the differential equation 


with the initial conditions x = 3, y = 1, for x = 3.05 and 3.10, using 
Euler’s method. 
Obtain the solution of the differential equation 


dye + 4x 


dx y2 
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with the initial conditions y = 1 when x = 1, for x = 1.1 and 1.2. 
Using Milne’s method, formulate the numerical solution of the 
differential equation 


assuming that yp, y;, yg, and y3 corresponding to x9, x1, Xp, and 
x3 are available, 

Using Milne’s method, formulate the numerical solution of the 
differential equation 


assuming that yo, y,. Yo, and y, corresponding to fy, t;, tg, and t, 
are given. 

Derive Eq. (6.51) from Eq. (6.50). 

Formulate the solution of the differential equation 


with the condition x = 0, y = 1, by the Fox-Euler method. 
Point out the difficulty encountered in using the Fox-Euler method 
in the solution of the nonlinear differential equation 


with the initial conditions x = x9, ¥ = yo. 

Formulate the solution of the above problem by Euler’s method of 
starting the solution. 

Formulate the solution of the second-order differential equation 


14. 


15. 


16. 


17. 


18. 


19: 


20. 


21. 


22. 


23. 
24. 
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with the initial conditions x = 0, y = 1 dy/dx = 2, by Milne’s 
predictor-corrector method. 

Formulate the solution of the above equation (Prob. 13) by direct 
substitution of central differences for the derivatives. 

Formulate the solution of the differential equation 


with the initial conditions x = 1, y = 1, by Noumerov’s method. 
Ax4 F 3 
Include the term 3 y{) in Eq. (6.90) and obtain the corresponding 


equation to Eq. (6.94) 

Evaluate the solutions given in Table 6-8 for h = 0.01 up to time of 
0.1. 

Evaluate the solutions given in Table 6-8 for h = 0.1 up to time of 
1.0. 

Formulate the solution of simultaneous equations of Prob. 2 by the 
Runge-Kutta method. 

Formulate the solution of differential equation of Prob. 13 by the 
Runge-Kutta method. 

Obtain the corresponding equations to Eqs. (6.106) and (6.107) 
using Kutta’s third-order rule of Eq. (6.98). 

Obtain the corresponding equations to Eqs. (6.106) and (6.107) 
using Kutta-Simpson three-eighths rule of Eqs. (6.100). 

Formulate the solution of Prob. 15 by the Runge-Kutta method. 
Write the proportional navigation problem of Fig. 6-2 in terms of 
velocities. In this case Eq. (6.108) becomes 


where V,, and Vp are the velocities of missile and target perpendicular 
to R. In this equation, Vp = Rw and 


Vy = Vo - Ro 
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Differentiating, we get 


dV dV 
ea ae wo 


dt dt 


or 


Ay = Ap - Ro - oR 


Carry out the rest of the formulation from Eq. (6.108) using the 
above relation. 

Draw the logic block diagram to represent the nonlinear function y 
as a function of x. 


Prob. 25 


Draw the logic block diagram to represent the nonlinear function y 
as a function of x 


Prob. 26 


27. 
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Write the logic of entering normally distributed white noise with a 
4 = 0.005 and a = 0.01 ft/sec? at point c of Fig. 6-2. 
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7 BOUNDARY VALUE 
DIFFERENTIAL 
EQUATIONS AND 
STABILITY OF SOLUTION 
OF DIFFERENTIAL 
EQUATIONS 


BOUNDARY VALUE PROBLEMS 


From the theory of linear differential equations it is known that the 
general solution of an nth-order differential equation will contain n 
arbitrary constants. These constants are evaluated from the given con- 
ditions of the dependent variable and its derivatives at certain values of 
the independent variable. As discussed in the previous chapter, if all of 
these conditions are specified for the value of the independent variable 
Xg, Where the solution is desired for x > xq, then the differential 
equation is termed an initial value problem. If some of the conditions of 
the dependent variable are specified at the end point x = L, where the 
solution is desired for xy S x < L, then the problem is termed a two- 
point boundary value problem. In this chapter the numerical solution 
of the boundary value problems will be considered primarily for linear 
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differential equations in order to compare results with analytical solu- 
tions. The methods described can equally well be applied to nonlinear 
differential equations. 

The methods of the previous chapter, together with relations given in 
Chap. 5, can be applied to the solution of boundary value problems. We 
will show that the numerical solutions of the boundary value problems 
can be accomplished either by iterative methods or can be reduced to the 
solution of a set of n simultaneous linear equations in n unknowns. In 
this case the number of equations will depend on the step size (delta) of 
the independent variable. The methods of numerical solution of linear 
simultaneous equations discussed in Chap. 1 can readily be applied to 
evaluate the unknowns of the simultaneous equations. 


Iterative Methods 


Consider the differential equation 


a™ 
i flys. D ones yD (7.1) 


where the superscripts refer to derivatives with respect to x, The above 
differential equation will require m auxiliary conditions for solution. 
Using finite-difference representations, Eq. (7.1) can be put in finite- 
difference form: 

Vig = FG Ip g Ye-a Vi- 1 Vir AD (7.2) 
where y;,, is expressed in terms of its values at previous x positions and 
the increment Ax. Since in a boundary value problem the values of y and 
its derivatives at x = xy and x = x, are given, Eq. (7.2) is solved by 
iteration as explained below. 

In order to start Eq. (7.2), several values of y, depending on the 
order of the differential equation (7.1), at x = x9, x = x9 + Ax, x = Xp 
4 2Ax,..., are needed. In boundary value problems not all of these 
values can be obtained from initial conditions given at x = x9 since some 
of them should satisfy conditions at the other boundary x = x,. In 
using iteration methods for the solution, all of the values of y for starting 
Eq. (7.2) are either taken from initial conditions at x = xg or are 
assigned trial values. Then Eq. (7.2) is applied successively up to 
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solving for y,,, which is the value of y at the boundary x,,. If the values 
Of Yn» %n- 1) Yn -g» +++» match the finite difference representation of the 
boundary condition at x = x, the solution is terminated; if not, the 
trial values of y for starting Eq. (7.2) are altered, usually linearly inter- 
polated as a function of values of y at x, boundary, and Eq. (7.2) is 
reapplied from x = x9. 


Method of Simultaneous Equations 


This method involves the direct substitution of the differential equation 
by an equivalent finite-difference equation and the solution of the 
resulting simultaneous equations. The general equation representing 
Yixy at x;,1 = Xp + 7 Ax (where xp is the boundary and i is the x index) 
is written in terms of previously available Vp Vio Vinge ee The 
boundary conditions at two boundaries, x= x9 and x = x,, are also 
formulated in finite-difference form and incorporated with the general 
Y;41 equations. The final formulation should result in 7 equations in / 
unknowns, where / is a function of the number of increments of Ax in 
the region x) Sx <x, and varies as a function of the order of the 
differential equation and the given boundary conditions. The methods of 
solution described above are used in the following example. 

Example. It is required to find the solution of the differential equation 


d?x 
dt? 


+ 3x = 1 (7.3) 


with the initial and boundary conditions (¢ = 0, x = 1) and(t = 1, x = D. 
The solution of this differential equation satisfying the given boundary 
conditions can be obtained from methods of the Appendix or texts on 


differential equations [1]. This solution is given as 
x= (2/8) - cosv3 in V3 + cosv3t+ + (7.4) 
siny3 3 


Equation (7.3) can be replaced by its difference form using central dif- 
ference relations of Table 5.4 of Chapter 5. Since the solution is desired 
for the interval t = 0 to ¢ = 1, Fig. 7-1(a), this interval is divided into n 
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figure 7-1(a). Increments for the solution of Eq. (7.3). 


equal parts resulting in AT = 1/n. Note that the value of x at t = 0 is 
Xo = 1 and at ¢ = 1 is x, = 1. Let us now replace the differential 
equation (7.3) with its central difference equivalent. For t = t, 


Xq — 2x, + x. 

ee ory = tj (7.5) 
AP 

fort=#@i= 2,3, 4,...,n- 2) 

X41 — 2%; + X44 


A? 


w8ey 28 (7.6) 


i i 


forts ty 


x - x x 
n-2 Tce aia (7.7) 


+ 8x = 
A? 


Iterative Method. The general form of Eq. (7.6) can be used to solve 
for x; , 1 as 
Xigy = ~My + 4; (2 - 3At?) + (An? t; 
(7.8) 
for F = 1,:2;...,%—-1 


where t; = i At. 
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Note that the values of xg and x, are known from the initial (1 = 0) 
and boundary (t = 1) conditions. The iterative method of solution starts 
by solving Eq. (7.8) fori = 1 


Xp = —Xq + x, (2 ~ 3At?) + (Ant, (7.9) 


where t, = At. Since the value of x9 = 1 is given in the problem, a value 
for x, should be assumed. Let At = 0.2; this will result in n = 1/0.2=5 
increments. Using this value of At and assuming a trial value of x, = 1, 
Eq. (7.9) can be solved for x. and subsequently Eq. (7.8) can be solved 
for x3, x4, and x;. This will result in 


%y = 1 
x, = 1 
X_ = 0.888 
(7.10) 
X3 = 0.68544 
X4 = 0.424627 
x5 = 0.144859 


Since the value of x, = x, from the boundary condition should be 1.0 
(higher than 0.144859 obtained above), let us assume another trial value 
for x, = 2. This results in 


X% = 1 
x, = 2 
X_ = 2.768 
CELI 
X3 = 3.21984 
X4 = 3.30930 
x5 = 3.03364 


Now, we passed the boundary value at x;. At this point we can use a 
linear interpolation between values of relations (7.10) and (7.11) to 
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obtain the next trial value of x,; that is, 


Peal 
| ee ee ee 
3 (crews = aa) Vine) 


= 1.29602 


where the parentheses on the right contain the desired value of x, at the 
boundary and its value from relations (7.10). Using the above value of 
x, and reapplying Eqs. (7.9) and (7.8), we get 


Xy = 1 
x, = 1.29602 
X_ = 1.44452 
(7.13) 
x3 = 1.43568 
X4 = 1.27855 
x, = LO 


which results in the desired value of boundary condition x,. 

Simultaneous Equations Method. Since the end conditions x9 and x, 
are given, x, through x, _ ; are the unknowns. Setting the value of xy = 1 
in Eqs. (7.5), writing Eq. (7.6) fori = 2, 3, 4,..., — 2, and setting the 
value of x, = 1 in Eq. (7.7), we can write n - 1 equations inn - 1 
unknowns x, through x, _, as given in Table 7-1. For a given value of 
At the value of n = 1/At can be calculated and the equations of Table 
7-1 can be solved. These equations can be written in matrix notation as 
follows: 


(A)(Q) = (Y) (7.14) 


where (A) is the square matrix of the coefficients of x, through x, _ ,, (@) 
is the column matrix of x’s, and (Y) is the matrix of the constants given 
on the right side of Table 7-1. From Eq. (7.14), the solution (Q) can be 
obtained by premultiplying both sides of the expression by (A)~! 


(Q) = (A)-1(Y) (7.15) 


TeV) oat Sievert Amt a () a0.) 
zee 4 = One tGIVe Peace ee OR ee tS See ee @> “7 104) 
z0v) es = ¥x + ive + Z-)ex + ae: 
zy) 22 =e ee (1 40g) ----- +--+ <= bx + (gtVE + e-y%x+ lx 
T-g0W = 0 wee e=------ (4) aog) ----------- x + give + Z-)'x 


(€"Z) by 40 uoHejUaseidey souaseyyIG ayuIy “L-Z age) 
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The solution was evaluated with a digital computer program to be 
discussed shortly. The results of the solution for various values of At 
are given in Table 7-2. It is noted that no significant change in the 
solution appears as the value of At is increased from 0.05 to 0.1 to 0.2. 
The analytical solution, evaluated from Eq. (7.4), is also given in Table 
7-2. 


Computer Program for the Solution of Eq. (7.3) 


The following computer program has been written in BASIC program- 
ming language [2] for the numerical evaluation of the above solution. 
In the program, matrix methods are used to solve the simultaneous 
equations given in Table 7-1. In the BASIC programming language rows 
and columns of matrices are started from zero; thus the first element of 
the matrix is the (0, 0) element, and so on. 


Table 7-2. Results of the Solution of Eq. (7.3) 


x x 
t From if | 

Eq. (7.4) At=0.05 At=0.1 At= 0.2 
0 1 1.0 1.0 1.0 
0.05 1.08541 1.08549 
0.10 | 1.16281 1.16297 1.16338 
0.15 1.23174 1.23198 
0.20 1.29182 1.29212 1.29287 1.29602 
0.25 1.34271 1.34307 
0.30 1.38416 1.38456 1.38556 
0.35 1.41599 1.41643 
0.40 1.32808 1.43854 1.43969 1.44453 
0.45 1.45039 1.45087 
0.50 1.45295 1.45343 1.45462 
0.55 1.44588 1.44635 
0.60 1.42933 1.42979 1.43091 1.43568 
0.65 1.40358 1.40401 
0.70 1.36893 1.36932 1.37028 
0.75 1.32576 1.32611 
0.80 1.27454 1.27483 1.27554 1.27856 
0.85 1.21576 1.21598 
0.90 1.14999 1.15014 1.15053 
0.95 1.07785 1.07792 
1.00 1.0 1.0 1.0 1.0 
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10 LET T=0.1 
20 LET N=1/T 
30 LET M=N-2 


32 DIMA (25,25),¥ (25,0) ,B(25,25),Q(25,0) 
35 LET Z=-2+3*T*T 

40 MAT A=ZER(M,M) 

45 MAT Y=ZER(M,0) 

50 LET A(0,0)=Z 

55 LET A(0,1)=1 

60 LET Y(0,0)=Tt3-1 

65 FOR J=1TOM-1 

70 LET A(J,J-1)=1 

75 LET A(J,J)=Z 

80 LET A(J,J+1)=1 

85 LET Y(J,0)=(J+1)*T*(Tt2) 

90 NEXT J 

95 LET A(M,M-1)=1 

100 LET A(M,M)=Z 

105 LET Y(M,0)=(M+1)*T*(Tt2)-1 
106 MAT PRINT A; 

107 PRINT 

108 MAT PRINT Y 

110 MAT B=ZER(M,M) 

115 MAT B=INV(A) 

120 MAT Q=ZER(M,0) 

125 MAT Q=B*Y 

127 PRINT“ANSWER MATRIX Q” 
128 PRINT 

130 MAT PRINT Q 

140 LET K=31.5 

145 FOR D=0TO 1STEPT 

150 LET X=(((2/3)-COS(K))/SIN(K)) *SIN(K*D)+COS(K*D)+D/3 
155 PRINT ”“D= “D,"X= “X 
156 PRINT 

160 NEXTD 

170 END 


Statement 10 of the program specifies the value of At = T = 0.1. 
Statement 20 computes the number of increments N = 1/T= 10. Since 
Table 7-1 has N — 1 equations, the number of equations will be 9. As 
mentioned previously, the matrix notation of BASIC language starts 
from the zeroth row; this will mean that equations are numbered from 
zero to 8. Or, by statement 30,M = N - 2=8. Statement 32 reserves 
computer memory space for matrices used in the program. Statement 
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35 computes the diagonal coefficients of x,, x9, X3,.-. of Table 7-1. 
Statements 40 and 45 set the elements of matrices A and Y of Eq. (7.14) 
equal to zero, Statements 50 through 105 compute the elements of 
matrices A and Y as given in Table 7-1. Statements 106 through 108 
print matrices A and Y. This is done for information only and, of 
course, does not affect succeeding results. The printout of these matrices 
is shown below. 


-1.97 1 0 0 0 0 0 0 0 
1 -1.97 1 0 0 0 0 0 0 
0 1 -1.97 1 0 0 0 0 0 
0 0 1 -1.97 1 0 0 0 0 
A= 0 0 0 1 -1.97 1 0 0 0 
0 0 0 0 1 -1.97 1 0 0 
0 0 0 0 0 1 -1.97 a 0 
0 0 0 0 0 0 1 -1.97 1 
0 0 0 0 0 0 0 i -1.97 
-0.999 
0.002 
0.003 
0.004 
Y =] 0.005 
0.006 
0.007 
0.008 
-0.991 


Statement 110 sets elements of matrix B equal to zero. Statement 115 
inverts matrix A and sets this inverse equal to B, that is, B = Aq}. 
Using this together with Eq. (7.15), the matrix containing the answers 
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will be 
Q = BY 


given by statement 125. Statements 127 through 130 print out matrix 
Q. The rest of the statements — 140 through 160 — compute and print 
out the analytical solution given by Eq. (7.4). Note that statement 
150 corresponds to algebraic Eq. (7.4). The results of this program are 
shown in Table 7-2. 

From the solutions discussed above it can be seen that the iterative 
solutions are more adaptable to hand calculations and where computer 
routines for the solution of simultaneous equations are not available. 
From Table 7-1 it is seen that the boundary value problem can be re- 
duced to the solution of a set of tridiagonal simultaneous equations. 
The recommended method of solution of this type of system as dis- 
cussed in the next chapter is Thomas’ method. However, if the number 
of equations is small, the use of other methods will not result in 
excessive computer time. 

As was seen in the case of the foregoing example and as will be seen 
in the case of the following example, the iterative method of solution 
requires some judgment in “bracketing” the starting values (x, in the 
case of the previous problem). Translating this into a computer program 
may require an excessive number of logical operations. 


Other Types of Boundary Conditions 


In the above example the values of the function x were specified at 
the end points t = 0 and t= 1. In a number of problems the values of 
the derivatives of the function may be specified at the end points 
instead of the values of the function itself. Let us again consider Eq. 
(7.3) as repeated below: 


a 
a + 3x = ¢t (7.16) 


but this time with the boundary conditions (¢ = 0, x = 1) and (t = 1, 
dx/dt = 1). Note that in this case the derivative at ¢ = 1 is specified. We 
again proceed with the formulation given in Fig. 7-1(a) and Eqs. (7.5), 
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(7.6), and (7.7), but introduce the derivative at ¢, = 1. Consider Fig. 
7-1(b), which is the end boundary of the problem. In this case the 
fictitious point 1, , , is added. The value of the derivative at ¢,, can be 
written using central differences 


on Sorel 
(“) Stil = %n-1 (7.17) 
dt}, 4, 2A 
— e ~ 
Xn-2 Xn-1 Xn Xn+1 
es 
ot 7 SO ESS 


figure 7-1(b). Boundary value att, = 1. 


Iterative Method. The solution of the problem in this case will pri- 
marily involve the application of Eq. (7.8) for i = 1 through n instead 
of n — 1 as was the case in the previous example. This allows the calcu- 
lation of x, , ;, which is required in the derivative Eq. (7.17). Note that 
from the boundary condition the final value of the derivative should be 
unity 


= (7.18) 


Assuming a trial value for x, = - 1 and using At = 0,2 as was done 
previously, Eq. (7.8) will result in 
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Xp = 1 

x, =-1 

X_ = -2.872 

X%3 = —4.38336 (7.19) 
x4 = —5,84472 

x5 = -5.63271 

Xg = -5.20477 

D = 0.349858 


Since this value is short of the boundary condition at x;, we assume 


another trial value of x, = —- 2. This results in 
X%y = 1 
% = -2 
%_ = -4.752 
x, = ~6.91776 aaa 
x, = ~8.22939 
x5 = —8.52149 
X%g = -7.75101 
D = 1.19594 


Using relations (7.19) and (7.20) and interpolating for the trial value of 
x, for the next iteration, we get 


Eee) 
ee gs | sR) 
: [at = | — 


~ 1.76841 


Using this value for the next trial, we get 
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%y = 1 

x, = —1.76841 

X_ = -4.31662 

x3 = —6.33083 (7.22) 
x4 = -7.56134 

x, = -—7.85249 

Xg = -7.16134 

D = 0.999997 


This solution satisfies the boundary and initial conditions. 
Simultaneous Equations Method. Equating the slope from Eq. (7.17) 
to the given boundary condition (dx/dt) = 1 at ¢ = t, = 1, we get 


Xney = 20+ x, 4 (7.23) 


The set of difference equations for this problem will be identical to the 
set given in Table 7-1 except for the last equation of the set and an 
additional equation for t,, The last equation of Table 7-1 (for t, _ ,) 
becomes 


Seepage + SAM) +x, = ¢ _, (An (7.24) 


Note that in the above equation x, is an unknown. The equation for t, 
can be written 


%quy + X_(-2 + BAN +x, 4) = t, (Ad? (7.25) 


In this expression, the value of x, , ; can be substituted from Eq. (7.23), 
resulting in 


2x, 1 + x,(-2 + SAD = t, (At - 2At (7.26) 


Thus, the substitution of Eq. (7.24) in place of the equation for to 
of Table 7-1 and the addition of Eq. (7.26) for ¢,, will incorporate the 
new boundary condition in the problem. This will result in n equations 
in n unknowns, x, through x,,. 
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Use of Forward and Backward Differences in 
Incorporating Boundary Conditions 


The boundary conditions can also be expressed in terms of forward 
and backward differences. Consider the second-order differential equation 


2 
y= F2 - fey, (7.27) 
dx? 


with the boundary conditions (x = x9, dy/dx = yg) and (x = x,, y= y;). 
Denoting the values of y at discrete x intervals as y; and considering 
Fig, 7-2, Eq. (7.27) can be written in central difference form for point 


yo y1 12 Yi-1 yi yit+l Yn-2 Yn-1 Yn 
x0 1 x2 ha | sf xi+1 ana 2 n=) Xn 


figure 7-2. Use of forward and backward differences in boundary value problems. 


i. Using difference expressions from Table 5-4 of Chapter 5 for y” and y’, 
Vi = Loe toe be Wee 
LN ee Ea i(ss Vp eet) (7.28) 
Ax 


Transposing the unknown y’s to one side of the equation, we get 


B54 p¥p¥j-p A”) = w; (7.29) 


where g; is a function of the given variables and w; is a function of known 
values of x;. The above expression can be written for i= 1 toi =n-1 
corresponding to values of y for x, through x,, 


8, (Yor Vy Yq Ax) = wy 


82 (Vg Yor Vy Ax) = We 
Meats SDR erate Mees a (7.30) 


6-2 n-1) In-2) Yn 9) SX) = W, 9 


Bn-1%n Yn- 1 Yn-2) Ax) = w,_1 
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The above relations are (n — 1) equationsin n+ 1 unknowns yo, ¥1, ---, ¥,: 
The solution can be obtained only if two additional equations are given. 
These additional equations are derived from the boundary conditions. 

The expression for the derivative of the function at yg in terms of the 
forward differences of Table 5-3 of Chapter 5 results in 


dy il 
= = Dy, = —(—8y, + 49, - Yo) B 
a: ge 0 Le (7.31) 


Equating this to the given value of the derivative at the boundary 
x = 0, we get 


—8y + 4¥; — Yq = 2(Ax) 9H (7.32) 


The expression for the derivative of the function at y,, in terms of the 
backward differences of Table 5-2 of Chapter 5 results in 


dy ih 
@) = Dig = 5 (8n - Mn-1 + Mn 2) (7.33) 


dx}, 2Ax 


Equating this to the given boundary condition at x = x,,, we get 
By, — 49_-1 + Yn—g = 2lAxd yr’ (7.34) 


Equations (7.32) and (7.34) along with the set of equations (7.30) 
constitute (n + 1) equations to be solved for (mn + 1) unknowns yo 
through y,. Note that in the solution of Eq. (7.27) the differentials 
were always replaced by differences with errors of order 2 

Example. Consider the solution of the differential equation 


y= ty tx? (7.35) 


where primes denote derivatives with respect to x, subject to the initial 
and boundary conditions (x = 0, dy/dx = 1) and (x = 1, dy/dx = 0). 
Replacing the differential equation with central differences of e = 0(h?) 
from Table 5-4 of Chapter 5, we get 
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Ay St PALE chs bak Ee eee (7.36) 
Ax2 2Ax : ; 


Transposing the unknown y’s to one side, we get 
V4 (2 — Ax) + y;(-4 - 2Ax?) + y,_1(2 + Ax) = x,2(2Ax?) (7,37) 
Using Eq. (7.32) for the given derivative (dy/dx = 1) at x = 0, we get 
~By9 + 4y) - Yo = 2(Ax) (7.38) 
Using Eq. (7.34) for the given derivative (dy/dx = 0) at x = 1, we get 
3y,- 4n-1 + Mn-g = 9 (7.39) 


Combining Eq. (7.38), Eq. (7.37) for i= 1, 2,...,(m — 1) and Eq. (7.39), 
we get the set of equations 

—38y9 + 4y) - Yq = 2Ax 

(2 + Axdyg + (-4 — 2Ax?)y, + (2 - Axdyy = 2(Ax)?x,? 


(2 + Axdy, + (-4 - 2Ax)y, + (2 - Axdyg = 2(Ax)? x2 


(2+ Ax)y, 9 + (4 - 2Ax)y,_1 + (2- Axdy, = x? 


Yn-9- Wn-1+ YW, = 0 


Equations (7.40) constitute a set of (n + 1) equations in (n + 1) unknown 
Yo through y,. In the above equations, x; = i Ax. Equations (7.40) are 
solved for the unknowns for Ax = 0.05, Ax = 0.1, and Ax = 0,2. The 
results are given in Table 7-3. 


Solution of Simultaneous Differential 
Equations 


As mentioned in the previous chapter, a differential equation of nth 
order can be reduced to n first-order differential equations. Thus, a set 
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Table 7-3. Solution of Eq. (7.35) 


—————— —— 


x a 
Ax =0.5 Ax =0.10 Ax =0.20 
0 -2.11359 -2.11158 -2.10602 
0.05 -2.06499 
0.10 -2.01918 -2.01718 
0.15 -1.97618 
0.20 -1.93598 -1.92297 -1.92823 
0.25 -1.89858 
0.30 -1.86397 -1.86193 
0.35 -1.83214 
0.40 -1.80305 -1.80096 -1.79485 
0.45 -1.77669 
0.50 -1.75301 -1.75085 
0.55 -1.73198 
0.60 -1.71352 -1.71126 -1.70448 
0.65 -1.69760 
0.70 -1.68412 -1.68173 
0.75 -1.67302 
0.80 -1.66419 -1.66163 -1.65379 
0.85 -1,65754 
0.90 -1.65294 -1.65018 
0.95 -1.65027 
1.00 -1.64938 -1.64636 -1.63690 
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of differential equations containing higher-order derivatives can be re- 
duced to a set containing only first-order derivatives. The solution of 
sets of first-order boundary value differential equations is obtained using 


the procedures discussed above. 


Consider the set of first-order differential equations 


# 
" 


= f(xy, t) 


y’ = g(x,y, t) 


(7.41) 


where primes denote derivatives with respect to t, with the initial and 


boundary conditions (t = 0,x = x9) and (t= l,y = y,). 


The above 


expressions can be written in difference form using forward differences 
of order A from Table 5-3 of Chapter 5. This results in 
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iy = fxy Xp tp 
(7.42) 
be bd 
SS Ss g(x; Vp tp 
Collecting the unknowns x and y to one side, we get 
hx; 4 Xi Vp At) = w; (7.43) 


F(X Vip Ip AD = 2; 


where h and f are functions of the given variables and w and z are 
functions of the independent variables ¢ and At. Writing these equations 
for i = 0 toi = n —1 will result in 2n equations and 2(n + 1) unknowns, 
that is, x) through x, and yp through y,. Since the values of x, and y,, 
are given, these can be substituted into the equations, thereby reducing 
the number of unknowns to 2n. At this point, the set of simultaneous 
equations can be solved by methods discussed in Chap. 1. 

Example. Consider the second-order differential equation given in 
Eq. (7.35) and repeated below: 


yt = y ty 4x? (7.44) 


with the boundary conditions (x = 0, y’= 1) and (x= 1, y’= 0). Denoting 
y’ = 2, Eq. (7.44) can be reduced to two first-order differential equations 


OF a) 


P - (7.45) 
zZ=z+yt+x 


In the case of these equations, the corresponding equations to Eq. (7.42) 
become 


Visi 7 Vi 


Ax 
(7.46) 


Seale Sh 


Ax 
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This results: 


Yin — ¥1 ~— Anz; = 0 


(7.47) 
ernest Ax) Ay ta Anca.) 


Dividing the interval 0 < x < 1 into n equal increments Ax, Eqs. (7.47) 
can be written for i = 0 to i = n —1 for the unknowns yo to y,, and Zo 
toz, 


-% +9 = Axzy 
ST ae -Axz, ae 
= ~Axzy -0 
“Yn-1 Yn ~Ax2n 1 =0 
~Axyg +2) = (1+ Axd2g + (Ax) xq 
-Axy, —(1+ Adz, +29 = Ax(x,)? 
-Axy5 
~Axy, 1 ~(1+Ax)z,_1 = Ax(x,_ 1)? - z, 


(7.48) 


Setting the values z) = 1 and z, = 0 in the above equations from the 
given boundary conditions, we will obtain 2 equations in 2n unknowns 
Yo through y, and z, through z, _,. 

The solutions of these equations are obtained for Ax = 0.1 and Ax = 
0.2 corresponding to n = 1/0.1 = 10 and n= 1/0.2 = 5 intervals between 
x = 0 tox = 1.0. These results are presented in Table 7-4, and can be 
compared to the solution given in Table 7-3. Note that the solution of 
Eqs. (7.48) also gives the values of z = dy/dx in addition to the values 
of y. The difference between values of » between Tables 7-3 and 7-4 
comes primarily from the fact that the formulation of Eq. (7.46) con- 
tains an error of order h while formulation of Eq. (7.36) contains an 
error of order h2. 
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Table 7-4. Solution of Eqs. (7.48) 


: Ax=0.1 Ax=0.2 

y Zz Y Z 
0 -2.11814 10% -2.11859 1,0* 
0.1 -2.01814 0.888186 
0.2 -1.92932 0.776191 -1.91859 0.776283 
0.3 -1.85170 0.664878 
0.4 -1.78521 0.555195 1.76333 0.555822 
0.5 -1.72969 0.448193 
0.6 -1.68488 0.345043 1.65217 0.346320 
0.7 -1.65037 0.247060 
0.8 -1.62566 0.155729 -1,58290 0.157150 
0.9 -1.61009 0.072735 
1.0 -1.60282 0.0* -1,55147 0.0* 


*Inserted from boundary conditions. 


STABILITY OF SOLUTION OF 
DIFFERENTIAL EQUATION 


As discussed here and in the previous chapter, the numerical solutions 
of differential equations depend on the particular finite-difference repre- 
sentation of the equation and the increment of the independent variable, 
Ax or At. Correlating the stability and the accuracy of the solution of 
the difference representation with the size of the increment is an import- 
ant step toward the justification of the finite-difference representation 
(see stability discussion of the previous chapter). Unfortunately, this 
correlation can be obtained analytically in a limited number of cases. In 
many practical applications the correlation of Ax and At with the 
accuracy of solution is made by running particular finite difference for- 
mulations on the computer and examining the resulting solutions. 

Analytic methods of stability and accuracy determination of finite- 
difference formulations involve the use of difference-differential equa- 
tions, as described below. 


ANALYTIC SOLUTION OF DIFFERENCE- 
DIFFERENTIAL EQUATIONS 


The finite-difference formulations of differential equations can be 
represented in the form 


> 7 
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x(t + iT) + v,xit 4G = TT] + vg xlt + Gi - 2) 7) 
(7.49) 


tee + uxt) = f(t) 


where T is the increment in ¢ and i is an integer. Using the previously 
given notation 


Eix(t) = x(t + iT) (7.50) 


Eq. (7.49) becomes 
(Ei + vy Bin} + vg Bin? + see + vj)x(d = f(t) (7.51) 


Following [3], the complementary solution of the above equation x, 
can be obtained by solving 


Gis Bil vgwi- 2 4 + vx 0 =0 (7.52) 


The solution of Eq. (7.52) is obtained by assuming 
x,(t) = Cem (7.53) 


where C is an arbitrary constant, and m is a number to be determined. 
Substituting Eq. (7.53) in Eq. (7.52), we have 


Comt(omit ever DT yy, omli- DT yy v;)= 0 (7.54) 
Dividing through by Ce”! and substituting q = e™T, we get 


qi+viqi-14vpqi-2+--- +4, = 0 (7:55) 


(Note that this equation is identical to Eq. (7.52) with q’s replacing E’s.) 
Assuming that t = nT, where n is an integer, the solution (7.53) becomes 


%,(nT) = Ce™"T = Cq” (7.56) 


292 _ Digital Computer Methods in Engineering 


The roots of Eq. (7.55) will give the values of q to be used in the solution 
equation (7.56). The roots of Eq. (7.55) can be obtained by standard 
polynomial root finding methods discussed in Chap. 3. Three cases are 
encountered after the roots are obtained. They are: 

1. Distinct Roots. In which case the solution (7.56) becomes 


X_(nT) = Cyq," + Cogg” + Cgqg” +--+ + C;q;" (7.57) 


where 4}, 79, ---, 4; represent the roots of the polynomial (7.55). 

2. Complex Roots. In this case the roots will be complex conjugate 
as the coefficients of Eq. (7.55) are real (see Chap. 3). The conjugate 
roots 4), G5 will be in the form 


= a+b) Gg =a- dj (7.58) 
where a and b are constants and j2 = — 1. These roots can be written as 
a, = Rel? gy = ReW? (7.59) 


where R = va? + b? and @ = tan=!(b/a). The corresponding solution 
may be expressed 


x_(nT) = Cyqy" + Coady” 
; . (7.60) 
= Rn(C, ein? + Cye7in?) 


Substituting e*/”° = cosn0 + j sinnO in the above equation, the solution 


becomes 


x,(nT) = R"(C} cosnd + Cy sinnd) (7.61) 


where C; and C, are new constants. 
3. Repeated Roots. For repeated roots 4; = q9 = q the solution 
is expressed as 


x, (nT) = q?(C, + Con) (7.62) 
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The particular solution of Eq. (7.49), x,, can be obtained in analogy 
with the differential equation theory [1]. In obtaining particular solu- 
tions we assume several types of solution, depending on the form of the 
function f(t), in Eq. (7.51). This solution is then substituted in Eq. 
(7.51) and the constants are evaluated. The following types of f(#) are 
often encountered in practice. 

1. f(t) = Akt = AknT where A and k are constants. In this case a 
particular solution of the type 


x,(nT) = CANT (7.63) 


is assumed and this solution is substituted in Eq. (7.51). The resulting 
equation is then solved for the constant C. Thus, the complete solution 
of the differential equation becomes 


c= x, (nT) + x, (nT) (7.64) 


2. f(t) = At™ = A(nT)™ where A is a constant and m is an integer. 
In this case, a solution of the type 


x, (nT) = Cy(nT)™ + Cyo(nT™" 1 +--+ + Cy yy (7.65) 


is assumed and this solution is substituted in the difference equation 
(7.51). The values of C,, Cy, .--,C,,, 1 are computed by equating the 
coefficients of equal powers of n on both sides of the resulting equation. 

3. f(t) = sinAT or cosAT = sinAnT or cosAnT. In this case the 
exponential equivalent of these functions can be written and the solution 
can be obtained by the method of case 1 above. Alternatively, a solution 
of the type 


x, (nT) = C; sinAnT + Cy cos AnT (7.66) 


can be assumed and substituted in Eq. (7.51). The values of the con- 
stants C, and C, are obtained by equating the coefficients of sin AnT and 
cos AnT terms on both sides of the equation, respectively. 

From the solutions discussed above it can be seen that the comple- 
mentary solution (7.56) represents the transient part of the solution as 
q’s are raised to the number of the time step n. Thus, since q’s in finite 
difference formulations are usually functions of the time increment T, 
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the complementary solution may be unstable for some region of T values. 
On the other hand, the particular solution follows the “driving input” of 
the difference equation as seen from the three cases above. For this 
reason the stability of a specific finite difference solution may be 
determined from an examination of the complementary solution of its 
difference-differential equation. The regions of T values for a stable 
solution are chosen by an examination of the complementary solution of 
the difference equation. 

Example. Consider the error analysis of the finite-difference solution 
of the differential equation 


Oy dy = BE (7.67) 


with the initial condition ¢ = 0, y = 1. 

The analytic solution of the equation can be obtained by standard 
methods of solution of differential equations. See Appendix on differen- 
tial equations. The complementary solution is the solution satisfying 


dy + 2y = 0 
dt 
or 
(D+ 2y = 0 (7.68) 


where D is the derivative operator. The root of this equation is D = - 2 
and the complementary solution becomes 


y = Ce* (7.69) 
The particular solution is obtained by assuming a solution of the type 


y, = At+B (7.70) 


Pp 


and substituting in Eq. (7.67) 


A+ 2At+ 2B = 3t (7.71) 
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Equating like powers of t, we have 
=e ee) 
A= 5 B= 7% 


The total analytic solution of the differential equation (7.67) becomes 


y= Cos 8-8 (7.72) 


> 


Setting the initial condition t = 0, y = 1 in Eq. (7.72), we obtain C = 7/4. 
Or Eq. (7.72) with this value of C becomes 


y= Te + 3t = 3 (7.73) 


which is the complete analytic solution of differential equation (7.67). 
Equation (7.67) can be represented in finite-difference form as 


y, mek 
Se lias Bb, = BnT (7.74) 


where T is the increment in t, that is, T = At. Denoting 2T - 1 =a, 
Eq. (7.74) becomes 
(E + a)y, = 3(nT)T (7.75) 


The complementary solution is obtained by setting the right side of this 
equation equal to zero: 


(E+ ay, = 0 (7.76) 


The root of this equation is q = —a, and the complementary solution 
becomes 


Oncomp = C(-a)" (7.77) 
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From the above complementary solution it can be seen that for large 
n and a2 1 the solution will grow beyond bounds. Therefore, the 
complementary solution will be stable only for 


G= 27-1 < 1 
or 


<1 (7.78) 


The particular solution of Eq. (7.75) is obtained by assuming a particular 
solution of the type 


Onpare = A(nT) + B (7.79) 


and substituting in Eq. (7.75). This will result in 
A(n + 1)T + B+ aA(nT) + aB = 3(nT)T (7.80) 


Equating the coefficients of like powers of x on both sides of the 
equation, we get 


3T 
Ave 33 
lia 2 
5 (7.81) 
Bia ee ieee) 8 
(1+ @)2 : 


In the above, the substitution 1 + a = 2T is used. Thus, the complete 
finite-difference solution of difference equation (7.74) will be the sum 
of Eqs. (7.77) and (7.79) with the constants of Eq. (7.81) 


= = C(-a)" +3 es 
Y_, = y(nT) = C(-a)" + 5 (nT) Zl (7.82) 


The constant C is evaluated by using the initial condition t = nT = 0, 
y = 0, orn =0, y = 0. Using this in Eq. (7.82) will result in C = 7/4, or 


y(nT) = Za" 5 3 (nT) = 3 (7.83) 
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Note that -a = 1 ~ 27. Using this relation in the above, and subtracting 
Eq. (7.83) from analytic solution (7.73), we get (set t = nT in Eq. (7.73)): 


€ = y(t) - y(nT) = Fleant =—(1 = 27") (7.84) 


where ¢ is the error between the analytic and finite-difference solutions. 
Note that the particular solution does not affect the error term as it is 
canceled by the corresponding analytic solution. The exponential 
function in (7.84) can be expanded in Taylor series as follows: 


e 2 3 i 
(Gea? ( _ op, - = - .-) (7.85) 


Thus, the effective error of Eq. (7.84) is introduced by neglecting the 
second and higher powers of T in the difference solution represented by 
(1 —2T)". The values of percentage error (¢/y ) (100) (using Eqs. (7.73) 
and (7.74)) were evaluated for various values of t using T as a parameter. 
A plot of this is given in Fig. 7-3. 

Example. As an example of the above analysis for a second-order 
differential equation, consider the differential equation 


—+o*x = 0 (7.86) 
with the initial conditions t = 0, x = 1, and dx/dt = 0, The analytic 
solution of Eq. (7.86) is of the form 


x = Asinwt + BcoswT (7.87) 


where A and B are constants. The initial conditions will establish the 
values of A and B as follows 


A=0 Be=il1 
Using these, solution (7.87) becomes 


x(t) = x(nT) = coswt = cosn(wT) (7.88) 
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o 


n 


Solution y, Eq. (7.73) 


Percent error 


0 0.4 0.8 ey) Ts 


figure 7-3. Error in difference formulation and 
the solution of Eq. (7.67) vs. t. 


where ¢ = nT (nis an integer, and T is the time step) is substituted for 
calculations to be performed later. 

The difference representation of Eq. (7.86) using central differences of 
order T2 can be written 


x = 26 4K 
athe x, = 0 (7.89) 
T 


Shifting the index n by one unit, the above equation can be written 
Sep 1%q_, ort? — 24%, = 0 (7.90) 


In analogy to Eq. (7.52), the above becomes 
(E? + (w?T? - 2)E + Ix, = 0 (7.91) 
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The roots of this equation are 


272 _ 272 _ 9)? _ 
(wo? T2 — 2) + Vw? T? - 2 - 4 (7.92) 


2 


Gy» I = 


Considering the part under the radical, three cases will arise 
Case 1. (w2 T2 _ 2 > 4 or wT > 2 the roots q, and qp will be real 
valued and distinct. The solution will be of the type given by Eq. (7.57): 


x(nT) = Cyq," + Coao” (7.93) 


This is clearly a nonoscillatory solution and is unacceptable since the 
analytic solution (7.88) is oscillatory. 

Case 2. (w2T? - 2) = 4 or wT = 2 double root g, = qg= 49. The 
solution will be of the type given by Eq. (7.62): 


x(nT) = q"(Cy + Cyn) (7.94) 


This is again clearly a nonoscillatory solution, and is unacceptable for the 
same reason as in case 1. 

Case 3. (w2T2 — 2 < 4 or wT < 2 complex conjugate roots of the 
type given by Eq. (7.58) 


q a+b; 


7:95 
et, (7.95) 


with a = —(o2T2 — 2)/2 and b =(1/2)/4 - (w2T? - 9)*. The solution 


will be of the type given by Eq. (7.61) 
x(nT) = R"(C} cosnd + Cy sinné) (7.96) 


with 


(7.97) 
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The constants Cy and Cy can be obtained using the initial conditions for 
t= nT =Oorn=0. Forn =O and x = 1, Eq. (7.96) will give C) = 1. 
The initial condition on the derivative dx/dt at t= 0 can be incorporated 
by writing the difference equation for the derivative as 


x(n + DT) - x(n - DT) _ 0 (7.98) 
2T 


or 
x{(n + DT] = x[(n - 1)T] at n = 0 


or using Eq. (7.96) with R = 1, 


Cy cos(n + 1A + C3 sin(n + 1)6 
(7.99) 


= Cj cos(n - 16 + Cy sin(n - DO 


Setting » = 0 and equating the coefficients of sine and cosine terms, 
respectively, we get 


=0 (7.100) 


Thus the solution of difference equation (7.90) with the appropriate 
initial conditions becomes 


x(nT) = cosné (7.101) 


where @ is given by Eq. (7.97). 

Comparisons of solutions (7.101) and (7.88) will show that the value 
of T should be so chosen as to make 6 ~ wT. Note that w is given in the 
problem equation (7.86). Using Eq. (7.97) together with the values of a 
and b in terms of wT, several values of 6 as a function of wT were calcu- 
lated as given in the following table. From this table it is noted that 
values of T corresponding to wT of up to 0.2 seem acceptable. 
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& 
< 
2 


0 
-100042 
-200335 
-301137 
-402716 
-50536 
-609385 
-715142 
-823034 
-933531 
1.0472 
1.16473 
1.287 


rere wmruawawrni? 


Re 


PROBLEMS 
1. Formulate the solution of the differential equation 


d?x 5 dx 


—. 


dt2 dt 


Peon i 


subject to the initial and boundary conditions (t = 0, x = 1) and 
(t = 2, dx/dt = 3), by using central differences of order T? and 
dividing the interval of 0 < ¢ < 2 into n equal parts. 

2. Formulate Prob. | with the boundary conditions (¢ = 0, dx/dt = 0) 
and(t = 2, dx/dt = 1). 

3. Formulate Prob. 2 using forward and backward differences to rep- 
resent the boundary conditions. 

4. Formulate the solution of the differential equation 


subject to the condition y’ = y=Qwhenx=1forO<x< 1. 
5. Formulate the solution of the differential equation 
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subject to the condition y = 1, dy/dx = 0 when x= 1 forO< x < 1. 
6. Obtain the solution of the boundary value differential equation 


2 
ay, 10% 16x = 0 
dx2 dx 


with y = 1 at x =O and y =2 at x = 1, by the iteration method. 
7. Obtain the solution of the differential equation 


with dy/dx = y = 0 at x = 1, by the iteration method. 
8. Solve Prob. 7 for boundary condition y = 1 at x = 0 and y = 2 at 
x=1. 
9. Obtain the solution of Eq. (7.16) by the simultaneous equations 
method and compare with results given by relations (7.22). 
10. Obtain the solutions given for Ax = 0.2 of Table 7-3. 
11. Formulate the solution of the sets of differential equations 


dx ahi. 
— =--+-4+1 
dt t t 
yh Sigs 
dt t t 


subject to the conditions x = 0 when t= 0, y = 1 when t= 1. 
12. Solve the difference-differential equation 


(E2 _ 5E + 6)y = et 


13. Solve the difference-differential equation 


14. 


15. 


16. 


17. 
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(E2 — ly = 12 4 2t 


subject to the conditions y = 0 when ¢ = 0 and y = 1 when t= 1. 
Solve the difference-differential equation 


(E2 — 29E + 4)y = x2 


subject to the conditions y = 0 when x = 0 and dy/dx = 1 when 
x=0. 

Discuss the stability of the solution of the central difference for- 
mulation of 


by examining the complementary solution of the difference-differ- 
ential equation. 

Discuss the stability of solution of the central difference representa- 
tion of the equation 


using the complementary solution of difference-differential equation. 
Discuss the accumulation of error in the central difference formu- 
lation of the differential equation 


subject to the conditions y = 0, dy/dx = 1 when x = 0. The analytical 
solution of the above equation is 


x2 +C,x+ Cy 


ad 
See iG 


304 


18. 


19. 
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Discuss the accumulation of error in the forward difference formu- 
lation of the differential equation 


dx 
dt 


subject to the condition x = 5 when t = 0. The analytical solution 
of the problem is 


x = Cet 


Discuss the accumulation of error in the forward difference formu- 
lation of the differential equation 


subject to the conditions y = 1 when x = 0. The analytical solution 
of the problem is 


y = (x + C)e* 
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8 NUMERICAL 
SOLUTION OF PARTIAL 
DIFFERENTIAL 
EQUATIONS 


The numerical solution of partial differential equations can be a 
subject of extensive study by itself. There are several excellent texts 
and references written on this subject, including [1, 2]. The presentation 
of the subject given in the following pages should be considered only as 
an introduction rather than as a complete treatment. 

The discussions start by giving simple examples to illustrate the general 
concepts. The generalized forms of the solutions and addition methods 
of solutions are given after the introductory material. 

A second-order partial differential equation can be written in the form 


2 2 2 
Awe a B(x, y) oats + Ctx) 2 Len f (man fee ae = 0 
Ox2 axdy ay? ax dy. 


(8.1) 
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where A, B, C, and f are functions of the given variables. If f is a linear 
function of the variables in parantheses, the partial differential equation 
(8.1) is considered to be a linear partial differential equation. If f is a 
nonlinear function of the given variables, the partial differential equation 
(8.1) is considered to be a quasi-linear equation since the second-order 
derivatives d2u/dx2, J? u/dxdy, ..., appear in the first power only. The 
partial differential equation (8.1) is categorized depending on the rela- 
tionship of A, B, and C coefficients as follows: 


B? — 4AC < 0 elliptic equation 


I 


B2 — 4AC = 0 parabolic equation 
B2 — 4AC > 0 _ hyperbolic equation 


Examples of the above categories, often encountered in practice, are 
as follows: 


2 2 

Laplace’s equation Laer aa = 0 elliptic 
ax? dy? 
2 

Heat equation CEE =a ou parabolic 
ax? at 


(where a is a constant and x and ¢ are space and time coordinates). 


207u — d®u 
ae ee 
ax? at? 


Wave equation hyperbolic 


(where c is a constant and x and t are space and time coordinates). 

The above partial differential equations are written in the cartesian 
(x,y) coordinate system. These equations can also be written in other 
coordinate systems such as cylindrical and spherical. The following 
discussion, however, will be limited to cartesian coordinate system 
representations. 


Numerical Solution of Partial Differential Equations 307 
PARTIAL DIFFERENCE OPERATORS IN 
CARTESIAN COORDINATE SYSTEM 
Consider the function 
(8.2) 


z = f(x,y) 


where x,y are the cartesian coordinate system (Fig. 8-1), and the partial 


derivatives are denoted by 


2 2 2 

eee = = 0, a ts TE Dey 
ax ay axdy x2 dy? 
(8.3) 


Taking the point (0, 0) as the origin the partial derivatives of Eqs. (8.3) 
become (using central differences) 


figure 8-1. Cartesian coordinate system, 
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"1 


28D, 2 = 1,0 ~ 2-1,0 
WD? 29 = 24,9 - 229.9 + 2-1,0 (8.4) 
2h8 D8 zg = 29.9 ~ 221.9 + 22.1.9 - 22,0 


where A is the increment in the x direction. 
Similarly, denoting k the increment in the y direction, we have 


2k D, 29 = 291 - 20,-1 
Re D,?2z9 = 29.1 — 229.9 + 20,-1 (8.5) 


2h3 D,3zq = 29,2 - 22,1 + 220,-1 - 20,-2 


The expression for the mixed derivative D,, of the function z is 


obtained by applying the derivative operator Dy to the existing expression 
for D,. Thus 


ec 4) 
D,y29 = D,(D,) = 0, 24) 


ee Gis 21-1) = Cs eevee (8.6) 
Akh 


24.1 Sat — Su eed 
4kh 


The above expression in analogy to Eqs. (8.4) and (8.5) can be written 


4kh D.y 2 = 24,1 ~ 21,-1 ~ 2-1,1 + 2-1,-1 (8.7) 


SOLUTION OF LAPLACE’S EQUATION 


The Laplace partial differential equation, as previously given, can be 
written in cartesian coordinates as 


2 2 
Soi LT Lee (8.8) 


=e 
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Ay 
op G oS ; 
Ax 


figure 8-2. Solution of Laplace's equation, 


where V2 is the Laplace operator (not to be confused with backward 
difference operator). Considering Fig. 8-2 and using difference relations 
given by Eqs. (8.4) and (8.5) in Eq. (8.8), we get 


Uy — Quy + U u, — 2u, + wu. 
2 atta Us Dare aH (8.9) 
Ax? Ay? 


Setting the values of Ax = Ay = h, Eq. (8.9) becomes 
Uy + Up + Ug + Uy - dug = 0 (8.10) 


Using the above relation and assuming the values of u’s as unknowns, 
the solution of Eq. (8.8) can be obtained by solving a series of simul- 
taneous equations as illustrated in the following example: 

Example. Obtain the solution of Laplace’s equation for the geometry 
of Fig. 8-3. In this figure the values of u at the upper and lower boundary 
are set at 1,000, while the values of u at the right and left boundaries 
are set at zero. The values at the corners are arbitrarily set at 500 so as to 
make the discontinuity of approaching the corners from the x and y 
directions equal, i.e., the change from the nearest point to the corner 
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u=500 1,000 1,000 1,000 1,000 1,000 500 


u=500 1,000 1,000 1,000 1,000 1,000 500 


figure 8-3. Example problem. Solution of Laplace’s equation, 


from the x and y directions is 500. The square cross section is divided 
into six equal increments of length h in both x and y directions. This 
will give 25 points or 25 values of u to be calculated. Since the divisions 
of the cross section are symmetrical, only the values of u at the following 
points will be calculated: 


Uy) Ugy Ugy Ugy Ug, Ugy Uy, Myo, and uy 5 


Using Eq. (8.10), equations can be written for these unknowns. These 
equations are given in Table 8-1. Note that in obtaining the coefficients 
listed in the table the following relations, arising from the symmetry of 
the problem, were used: 


u 


Bye te eR Hoe hone gee ig nae ooy oe )) Siow 


The solutions of these equations by previously outlined methods of 
Chap. 1 were evaluated using a digital computer. The answers are 
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Table 8-1. Coefficients of Simultaneous Equations 


Constant 


-1,000 
-1,000 
-1,000 


as follows: 

u, = 500.0 

Uy = 682.692 

u, = 730.769 

ug = 317.308 

u, = 500.0 (8.11) 
ug = 557.692 

u,, = 269.231 

Uyy = 442.308 

uy, = 500.0 


Note that these values represent the steady-state temperature distribution 
in a homogeneous solid material if u is considered to be the temperature. 


ITERATIVE METHOD OF SOLUTION OF 
LAPLACE’S EQUATION 

The solution of Laplace’s equation (8.8) can be obtained by a simple 
iterative method due to Liebmann. This method is based on the fact 
that the value of wu at any point is the average of the four surrounding 
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values as given by Eq. (8.10), that is, 
Uy = rac + Ug + Ug + Uy) (8.12) 


The method starts by assuming an initial distribution of u over the 
region and calculating new values at each iteration using Eq. (8.12), The 
procedure is further explained through the following example. 

Example. Again consider Fig. 8-3 of the previous example, but this 
time let us use matrix notation to specify various points of the cross 
section, as given in Fig. 8-4. Using the same conditions at the boundaries 
as given in the previous example, and assuming a value of u = 500 for all 
of the inside points of Fig. 8-4, the matrix values of u at iteration zero 
will be 


0 1 2 3 4 3 6 + Column 


ANPRWNHHKO ae 
= 


The first iteration starts by averaging the four values around each of 
the inside points and replacing the old value with this average. The 
averaging starts with element (1, 1), (1, 2),..., (1, 5) and proceeds with 
(2, 1), (2, 2),..., and so on. Thus at the end of the first iteration the 
matrix of u values will be 


ITERATION 1 
500 1000 1000 1000 1000 1000 500 
0) 500 625 656.25 664.063 541.016 0 
0) 375 500 539.063 550.781 397.949 0 
ie) 343.75 460.938 500 512.695 352.661 0 
ie) 335.938 449.219 487.305 500 338.165 0 
10) 458.984 602.051 647.339 661.835 500 ie) 
500 1000 1000 1000 1000 1000 500 
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(0, 0) (0, 1) (0, 2) (0, 3) (0, 4) (0, 5) (0, 6) 
,000 ,000 1,000 1,000 ,000 500 
(1,1) (1, 2) ) (1, 4) (1, 5) 

Meee. |. bs bs | 


eae 


u=500 1,000 1,000 1,000 1,000 1,000 500 


(1, 6) 


(2, 6) 


(3, 6) 


figure 8-4. Solution of Laplace’s equation by iteration. 
Iterations can proceed in a similar manner resulting in the following 
values of u for the 10th and 20th iterations 


ITERATION 10 


500 1000 1000 1000 1000 1000 500 
i?) 500. 682.775 730.923 682.836 500.072 0 
0 317.225 500. 557.798 500.125 317.375 0O 
it) 269.077 442.202 500. 442.357 269.264 0 
0 317.164 499.875 557.643 500. 317.316 0 
it) 499.928 682.625 730.736 682.684 500. 0 
500 1000 1000 1000 1000 1000 500 
ITERATION 20 
500 1000 1000 1000 1000 1000 500 
9 500. 682.692 730.769 682.692 500. 0 
0 317.308 500. 557.692 500. 317.308 0 
0 269.231 442.308 500. 442.308 269.231 0 
0 317.308 500. 557.692 500. 317.308 0 
it) 500. 682.692 730.769 682.692 500. 0 
500 1000 1000 1000 1000 1000 500 
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Note that the values shown at iteration 20 are completely symmetrical 
and agree with the values of Eq. (8.11) to three significant figures. 

The following digital computer program was written in BASIC pro- 
gramming language [3] to accomplish the iteration procedure discussed 
above, 

Statement 5 of the program reserves space for the matrix u. State- 
ment 10 sets each element of matrix u equal to unity. Statement 15 
multiplies all of the elements of matrix u by 500, which is the initial 
value of u at iteration zero. 


5 DIM U(6,6) 

10 MAT U=CON 

15 MAT U=(500)*U 
25 FOR J=1TO5 
30 LET U(0,J)=1000 
32 LET U(6,J)=1000 
35 NEXTJ 

40 LET U(0,0)=500 
42 LET U(0,6)=500 
44 LET U(6,0)=500 
46 LET U(6,6)=500 
50 LET Z=0 

70 FORI=1TOS 
75 LET U(I,0)=0 

80 LET U(I,6)=0 


85 NEXT! 
87 PRINT “ITERATION”Z 
88 PRINT 


90 MAT PRINT U; 

92 LET 2Z=Z+1 

95 FORI=1TO5 

100 FOR J=1TO5 

105 LET U(I,J)=U(1+1J)+U(1-1,5)+U (1 J+1)4+U (1 J-1) 
107 LET U(I,J)=U(1,J)/4 
110 NEXT J 

115 NEXTI 

116 PRINT 

117. PRINT “ITERATION”Z 
120 MAT PRINT U; 

130 IF Z<10 THEN 92 

135 END 


Statements 25 through 85 enter the values of u at the four boundaries 
of Fig. 8-4. (Note that Z in statement 50 is the number of the iteration.) 
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Statements 87 through 90 print the matrix of u values at zeroth iteration. 
Statements 95 through 115 compute values of u using Eq. (8.12). State- 
ments 116 through 120 print out the answers. Statement 130 sets an 
upper limit to the number of iterations. (Note that statement 130 was 
changed to Z < 20 to obtain iteration 20 given above.) 


GENERAL FORM OF LAPLACE’S EQUATION 


The above formulations are specialized forms of a more general 
relation describing the finite-difference representation of Laplace’s equa- 
tion. In a number of more complicated applications the formulas 
described in the following pages are preferred to the ones given pre- 
viously as they enjoy a faster rate of convergence to a solution. 

The general form of Laplace’s equation can be written in two- 
dimensional form as 


2 
ae * f(x,y) (8.13) 


where a and § may be functions of variables x and y, with the condition 
of elliptic equations satisfied. The above equation is known as Poisson’s 
equation and specializes to Laplace’s equation for a = 6 = constant and 
f(x,y) = 0. The finite-difference representation of Eq. (8.13) for the 
equal increments Ax = Ay = h can be written 


y, u(x +h, y) + ygu(x - h, y) + ygulx, y + h) + ygulx, y - h) 


+ ygu(x,y) = g(x,y) (8.14) 


where y’s are functions of x, y, and the increment h. Solving the above 
equation for u(x,y) and denoting the resulting coefficients by v’s, we 
have 


u(x,y) = vyulx + hy) + vgulx - h, y) + vg U(x, ¥ + h) 
(8.15) 
+ vgulx, y — h) + 7(x,y) 
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In addition to the previously discussed solutions, several iterative proce- 
dures are available for the solution of the above equation. Assuming that 
the values of the variable u(x, y) are available at iteration n over the x,y 
region and denoting these values by u, (x,y), the following iteration 
procedures can be formulated. 

1. Gauss-Seidel method or the method of successive displacements. 
The iteration formula for this method is 


Un (Xs) = vyU_(x + hy y) + vou, , (x - h, y) + vgu,(x, y + A) 


+ Mgu, (x,y - h) + 7, (x,y) (8.16) 


In a given problem the initial values of the function up (x, y) are given 
along the boundaries, as was the case of Fig. 8-4. Assuming a set of 
values for inside points at iteration zero, Eq. (8.16) can be solved 
successively for points (1, 1), (1, 2), (1, 3),..., (2, 1),...and so on, at 
each point using new values of u as they become available. After getting 
to point (5, 5) of the figure the second iteration is started from (1, 1). 
Continuing this procedure the final solution of the problem is obtained. 
Note that Eq. (8.16) in the case of Laplace’s equation reduces to the 
previously discussed Liebmann method. 

Equation (8.16) can be slightly modified to improve the rate of 
convergence of the solution. This modification is as follows: 


4, (9) = wlvyu,(x + hy y) + vou, , (x - hy y) + vguy(x, y + h) 
+ Vg, 410%, y - h) + 7, (x,y)] - (o - Du, (x,y) 
(8.17) 


The parameter @ is known as the relaxation factor and the methods 
which use this equation are termed as successive-overrelaxation methods. 
Note that for » = 1 Eq. (8.17) reduces to Eq. (8.16). The choice of w 
determines the rate of convergence of the solution. No general expression 
for optimum @ (fastest rate of convergence to a solution) is available for 
use in Eq. (8.17). However, in the case of Laplace’s equation 7 = 0 and 
Vy = Vg = V3 = V4 = 1/4, Eq. (8.17) reduces to 
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cote ee E suas oe 
Ped) = oft fu, (i +Lp)+uny@-L)+u,Gj+D (8.18) 


+t, 416, j- Ut -@ - Du) 


where i and j are indices in the x and y directions. 
The optimum value of « for this case is given [1] as 


= 2 
@=1+ (4 ;) (8.19) 
+V1l-f 


where 


and a and » are the sides of the rectangle. The above expression for 
a = b=1 and small h becomes 


2,2 
eae 


ce cos 7h poae 
1+ sinzh, 1+ 7h 


Although the above value of » applies to the specialized case of Eq. 
(8.18), it is used as a guide in selecting values of in the solution of the 
more general Eq. (8.17). It can be shown with the o of Eq. (8.20) that 
the successive-overrelaxation method converges 2/7h times faster than 
the Gauss-Seidel Method. 

2.  Alternating-direction implicit method. This method, developed 
by Peaceman and Rachford [1], is along the same lines as the relaxation 
methods discussed above. The application of this method, however, 
involves solution of tridiagonal linear simultaneous equations at each 
stage of iteration. For this reason before discussing the method we will 
describe the solution of this type of simultaneous equations. 


Wt 


(8.20) 


& 
" 
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Consider the set of tridiagonal simultaneous equations 


Budget iG; To.= Dy 

j 8.21 

NG WE aT Dae MeNG Orzo) =e (Fear nh vaneg resi )s (MAD, 

ATs eal eee 

where A, B, C, and D’s are constants and T’s are the variables. The solu- 
tion T’s can be expressed 


T, = 4 
sd gies (GS 9 pe ee (8.22) 
T= gy = 57,1 
where 
ane a a D; - Aja;-4 
; By " B,- A;b, 
Gi = 2,3,4,...,m—- 1) 

Cc CG; 
bh =-— b; = ——_——_ 

By B; - A;b;_4 (8.23) 


The above method, which can be derived from gaussian elimination 
method of solving simultaneous equations, is due to Thomas [4]. Tri- 
diagonal simultaneous equations (8.21) arise quite often in numerical 
solutions of partial and ordinary differential equations. 

Getting back to the alternating-direction method of Peaceman and 
Rachford, the basic iteration equations of this method are, for 
Ax = Ay = h, 


Un 41/2059) = Uy (xy) + rlulx + hy y) + wlx — hy) ~ Qulx My ve 
+ rlu(x, y + A) + w(x, y - A) - Qu(x,y], 


Hy 41069) = Uy, q(t y) + ruta + hy y) + whe - hy) - Qulx yy yo 
+ rlulx, y + A) + ule, y - A) ~ Quy), 1 


(8.24) 
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where the subscripts refer to iteration number associated with the variable 
u and r isa constant. Assuming that the values of wu at iteration nu, (x, y) 
are available, each complete iteration will require the solution of the 
first and then the second of Eqs. (8.24). Mechanization of the first of 
the above equations will result in m tridiagonal simultaneous equations, 
where m is the number of points inside the rectangle. For example, in 
the case of the problem of Fig. 8-4, the first of Eqs. (8.24) will give 


Uy gol = u, (1D + r[u(1,2) + w(1,0) ~ 2uA,D], , 15 
+ r(u(2,) + u(0,1) — 2u(1,0)],, 


(1,2) = u,(1,2) + rlu(1,3) + wD ~ 2u(1,2)1, , 2 
+ r(u(2,2) + u(0,2) ~ 2u(1,2)], 


(8.25) 


= u,,(5,5) + r[u(5,6) + u(5,4) — 2u(5,5)), . ro 
+ r[u(6,5) + u(4,5) — 2u(5,5)],, 


Transferring the unknowns of the above equations to one side, we get 


Beery (Bd) ct Ga the 5 nym, C12) Ds 


Peele Bou. 11,2) Cou. 4 .(13) = D, (8:26) 


n+1/2 


Ags Un 4 1054) + Bog up, , 105.5) = Dog 


The solutions of the above equations by the procedure outlined in 
Eqs. (8.22) and (8.23) will result in values of u at iteration n + 1/2. In 
Eqs. (8.26), A, B, C, and D are known quantities which are functions of 
previously available u,,(x,y). After calculating the values of uw, , /. the 
second set of Eqs. (8.24) are used in much the same manner to compute 
u,.1- Note that the first set of Eqs. (8.24) represents a row iteration 
(x direction) and the second set represents column iteration (y direction) 
and hence the name alternating direction. The term implicit in the name 
of the method is used to denote the fact that solutions at iteration n + 1 
cannot directly be obtained in terms of solutions at iteration n as was 
the case with overrelaxation methods. On the other hand when solutions 
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at iteration n + 1 can be expressed in terms of solutions at n the method 
of solution belongs to the general category of explicit methods. 
It is suggested [1] to use the equation 


1 
(bx - 1) 


aW/it-1) 
x =(% 
() 


for determining the values of the constant r. In the above equation a 
and b are the sides of the rectangle, k is the number of iterations, and t 
is the number of the last iteration. Since t is not known at the beginning 
of a given problem an assumption as to its value is made and the resulting 
r’s are computed. These values are then used in Eqs. (8.24). If the 
number of iterations ¢ seems unsatisfactory from the convergence view- 
point of resulting solutions, another number is chosen and the calculations 
repeated. 

In general, it can be shown that the number of iterations needed to 
achieve specific solution accuracies is much less using the alternating- 
direction method than successive overrelaxation. However, this apparent 
advantage should be balanced against the difficulty of implementing 
calculations depicted by Eqs. (8.25) and the selection of parameter r. 


Pe V2. Byres t 


™) = 


(8.27) 


SOLUTION OF THE HEAT EQUATION 


Consider the one-dimensional heat conduction equation 


Er tad (8.28) 


where u(x,t) is the temperature distribution, x is the space dimension, 
and ¢ is the time. The constant a depends on the heat transfer properties 
of the solid in question. For simplicity, assume that a = 1. Using the 
relations 
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au Yis15 ~ iG + Mi-1,j 


Ox? h? 


where h and T are the space and time increments Ax and At, respectively, 
Eq. (8.28) becomes 


Ys j41 = Mp4 1,7 + OL — Qedu; 5 + omy; (8.29) 


Indices i and j are used for space and time dimensions, respectively. a in 
Eq. (8.29) is related to h and T by 


4 ue (8.30) 


Having the values of the temperature u at time jT, u(i, j) for all i’s, the 
values of the temperature at time (j + 1) T, u(i,j + 1), can be obtained 
using Eq. (8.29). This equation takes an especially simple form when 
a= 1/2, that is, 


1 
Wage = obj 15 t Yin 1p (8.31) 


Or, the temperature at a point i at time j + 1 is equal to the average of 
the temperatures at either sides of i, i+ 1, and i-—1, at time j. It can also 
be shown that the solution represented by Eq. (8.29) is stable only for 
values of a < 1/2 (see [2] and the following general discussion). The 
numerical solution of the heat equation (8.28), using the above method, 
is illustrated by the following example. 

Example. Consider the solution of the temperature distribution 
equation 


a2 u(x, ) ia du(x, t) (8.32) 
ax? ot 
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with the following initial and boundary conditions: 


u(0,t) = 100 constant temperature boundary at x = 0 


ga) )=0 insulated boundary at x = 1 


Ox 


u(x,0) = 0 initial temperature equal to zero 


In order to use matrix notation, the increments and indices are set up as 
in Fig. 8-5. The insulated boundary condition 


du(nh,iT) _ ga ij (8.33) 
ox 


where T is the time increment, can be represented by using backward 
differences on the space coordinate x at x = nh 


Ynj~ Yn-i _ 9 (8.34) 
Ax 


or 


(7 ir top Pe 


That is, when the temperatures of the nth and the (mn — 1)th rows of 
Fig. 8-5 are equal, there will be no flow of heat between the regions 
(n-1)h <x nh, 

Assuming that the region between x = 0 to x = 1 is divided into ten 
equal increments of h = 0.1, the value of n = 1/0.1 = 10. The initial and 
boundary conditions now become 


Ujg = 0 ame he pooe eH 
ig = ean 
uo. = U j= 492, 8) 105 
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figure 8-5. Solution of heat flow equation (8.32). 


Having these values, Eq. (8.29) can be solved for u; ;,, for j = 0 and 
i=1,2,3,4,...,9. The value of uj9 9 is set equal to ug 9 by the last of 
Eqs. (8.35). Having these values, Eq. (8.29) can now be used to evaluate 
u’s at the next time step by setting j = 1 andi=1,2,3,..., 9. The value 
of Ui9.1 = Ug This procedure can be continued for j = 2, 3,..., and so 
on, 

Since the value of h was set at 0.1, a value of T= 0.005 will result in 


T _ 0.005 
n2 0.01 


a= 


Using this value of a, solutions can be evaluated for t = 0.005, 0.01, 
0.015, and so on. These solutions were evaluated by the digital computer 
program given below, and the results are presented in Fig. 8-6. It is 
noted that as time increases, the solution approaches the steady-state 
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Termperature t 


0 0.1 0.2 0.3 0.4 0.5 06 0.7 0.8 0.9 1.0 
Distance x 


figure 8-6. Solution of heat equation. 


value of 100. The digital computer program was written in BASIC 
programming language [3] to accomplish the above numerical procedure. 

Statement 10 reserves memory spaces for u. This statement assumes 
that there are ten divisions in the x direction and that the time extends 
for 160 increments. Statement 11 sets the value of the index M (time 
index) equal to 160. Statement 15 sets matrix U equal to zero. State- 
ment 20 sets the value of A = a = 0.5 for an h = 0.1 and T = 0.005. 
Statement 25 sets the value of index N = 1/h = 10. Statements 40 
through 85 implement the first two equations of (8.35). Statements 90 
through 120 carry out the numerical solution using Eq. (8.29) and the 


10 DIM U(10,160) 
11. LET M=160 
15 MAT U=ZER 
20 LETA=.5 

25 LET N=1/.1 
40 FOR I=1TON 
50 LET U(1,0)=0 
60 NEXTI 
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70 FOR J=0TOM 
80 LET U(0,J)=100 


85 NEXTJ 
90 FOR J=-OTOM-1 
94 LET K=J+1 


95 FOR I=1TON-1 

100 LET U(I,K)=A*U(I+1,J)+U(I,J)*(1-2"A)+A*U(I-1,J) 
115 NEXTI 

116 LET U(N,K)=U(N-1,K) 
120 NEXT J 

132 FOR J=0 TO M STEP 20 
133 PRINT “J="J 

134 FOR I=OTOM 

135 PRINT U(I,J); 

136 NEXT! 

137 PRINT 

138 NEXT J 

140 END 


last of Eqs. (8.35). Statements 132 through 138 print the answers which 

are the elements of matrix U. These answers are printed for J = 0 (t = 0), 

J = 20 (t = 0.005 x 20 = 0.1), J = 40 (t = 0.2),..., = 160 (t = 0.8) 
starting with i = 0, 1, 2,..., 10, as shown in Table 8-2. Note that the 

first value of U for each J is equal to 100, which is the boundary 

condition at x = 0. 


GENERAL FORM OF THE HEAT EQUATION 


The general form of the heat equation in one space dimension can be 
written as 


—-o—=0 o = const >0 (8.36) 


The boundary conditions u(x, ?) and the initial condition u(x,0), are 
assumed to be available. This equation can be written in the following 
finite-difference form: 


m+l_ym 9 (82u)."*) + (1 - @)(52u),” 
2 Bie pee iene a eG (8.37) 


At (Ax)? 


TEee'ss TEE6S8 LIE 98 


9ELOL8 SE8T 88 S19°68 90EE 16 8187 °€6 LIv'S6 LOLI'L6 oor 
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LL86°78 l6brrs IZEE98 806$°88 8ST16 L896'€6 77696 oor 
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TSE9N'SL TSE9'SL TIOE'9L 

960S°LL SES'6L 601078 6S86'P8 TT9E'BB TE90°76 £SL6'S6 oor 
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where i is the space and n is the time index (x = i Ax, t = n Ad) and the 
6? operator is used to designate 


ewe =e — aus ue, (8.38) 


and @ is a positive constant between zero and one. Thus, Eq. (8.37) in 
the case of @ = 0 specializes to 


uptt— un — (**) (3,1 - 24" + uP, 4) = 0 (8.39) 
Ax 


Using the notation 


AL 
Ax2 


a=0 


Eq. (8.39) becomes 


ut) = au?,, + (1 - Qadu;" + au?_, (8.40) 


i i+1 i 

This equation, but for indexing, represents Eq. (8.29). For any other 
value of @ (@ #0) Eq. (8.37) will result in a set of simultaneous equations 
to be solved at each iteration. The condition for the stability of solution 
of Eq. (8.37) is, from [2], 


2oAt. 1 
ne) 1-28 


ifO<a< 3 
(8.41) 


No restriction if 


It can be further shown that the error in representation (8.39) is of the 
order 


e(u) = O(At) + O[(Ax)?] (8.42) 
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By varying the values of @ in Eq. (8.37) a number of relations can be 
obtained for solutions u(x,t) (see [2]). One such solution is for 6 = 1/2 


yet! _ ym (8?) ,” - (82 u),”*? 


: Ll oie (8.43) 
At 2Ax2 


developed by Crank and Nicholson in 1947. It can be shown that this 
solution is always stable (0 < @ < 1) and the truncation error is of the 
order of magnitude 


e = O[(AD?] + O[(Ax)?] (8.44) 


Expansion of Eq. (8.43) will result in 


a At 
upt)—u” = oats ~ 2u; + u;_ 4)" (8.45) 


+ [u;,1 ~ 2u;, + u;_.I"*}) 


Noting that the values of u at time step n are available, the above 
equation can be put in the form 


Avutt) + Buftls Cutt} =D, i = 2,3,...,M-1 


i i i 


(8.46) 


where M is the number of points in the x direction. And, since the 
boundary conditions are known, the equations representing the bound- 
aries will be of the form 


iBT ie A Cutt = aD 
(8.47) 


n+ 


nel 1 
Ayu + By uty 1 


Dy 


The solution of Eqs. (8.46) and (8.47) can be obtained by the method 
of Eq. (8.21), at each iteration. Note that in formulations where @ 4 0 
Eq. (8.37) results in a set of simultaneous equations to be solved at 
each iteration. 
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Formulas similar to Eq. (8.37) can be derived for the solution of the 
two-dimensional heat equation 


ae 1G (8.48) 


where A and C may be functions of x and y. Denoting 


ur i = uli Ax, j Ay, n At) (8.49) 
and 

mn Af c 

a tg i; + 4-1) qa tise ~ uf + ¥Ej-1) 

(8.50) 

We can write the finite difference representation of Eq. (8.48) by 

urtl_ yn, 

ij De apn t} She Ag? (8.51) 


At 


where again @ represents a number between zero and one. For @ > 0 
the above equations result in a system of simultaneous equations to be 
solved at each iteration. For @ = 0, of course, this is not required as the 
equations for u”*! will be in terms of uv” known quantities. The con- 
ditions of stability of solution of Eq. (8.51) are 


23) ee ee 
(Ax)2 (Ay)? 2 — 40 
No restriction if a OS 1 


This can be considered as generalization of stability condition found in 
the one-dimensional case. 
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SOLUTION OF THE WAVE EQUATION 


The wave equation, which represents the position of a vibrating string 
as a function of time (Fig. 8-7), can be represented as 


2 2 
iat Re (8.52) 
ax? at” 


where y(x, #) is the vertical position at distance x and time ¢ and c? isa 
constant depending on the physical properties of the string. 


y (x,0) = F(x) 
58 dy (x,0) = 0 
iq or 
| y 
x= 0 ae x=L 
y(0,1) = 0 ae y(L,t) = 0 


x 


figure 8-7. Coordinates of wave equation. 


Assume the following initial conditions for the wave equation (8.52) 


y(x,0) = f(x) 

(8.53) 
9 (x0) = 0 
at 


These equations give displacement and velocity of the string at time 
zero. The assumed boundary conditions at x = 0 and x = L, where L is 
the length of the string, are as follows: 


y(0,) = 0 
y(L,t) = 0 


(8.54) 


These equations state that the end points of the string are fixed. Let us 
denote the deflection at x; at time t; by y;,;, and the space and time 
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increments by A and T, respectively. Having done this, Eq. (8.52) can be 
written in central difference form (of order h?) as follows: 


1 
Mig — Mig * %i-17 = Vv — Mig + Yai-v (8.55) 
a 
where 
2_ ce T2 
h2 


The yalue of y; ;, ; can be obtained in terms of values of y at the previous 
time steps from Eq. (8.55). This results in 


Vajer = 2° 544,5 + Yi-1,p + 20 - 09,5 - HE 5-1 (8.56) 


For a = 1 (in [2] it is shown that solutions will be stable only if @ <1), 
this equation reduces to the simple form 


Sete OTs F 1-1p — Viger (8.57) 
Let us assume that the length of the string is divided into n parts 


where n = L/h. The initial and boundary conditions (8.53) and (8.54) 
can now be written 


Yio = Fix) i= Oton 
e179; 
pL 0), %pey = %i1, | = Oton 
2T i ae (8.58) 
¥o,; = 9 jae 2s as 
AP j= 1,2,3,... 


In the second of the above equations, central differences are used to 
represent the initial zero velocity condition. 
Equation (8.56) for any i and j = 0 can be written 


Vina = 070i 41,0 + Yi-1,0) + 20 - a) 95,9 - 94-4 (8.59) 
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The second of Eqs. (8.58) states that y; _; = ¥; 1; substituting in Eq. 
(8.59), we get 


2 
a 
aimee acento + ¥;-1,9) + = «9; 4 (8.60) 


This equation can be used for obtaining the value of Via ford ='1,2; 35.255 
n-—1. Note that the values of Yo.1 and y,, ; are given by the boundary 
conditions (8.58). Having these values, and setting j = 1 in Eq. (8.56), 
the values of y; 9, i= 1, 2,3,...,—1 canbe obtained for t = 2At = 27. 
The procedure can be carried out for j = 3, 4, 5,..., and so on. 

Example. As a numerical example of the above equations, consider 
the problem of the vibrating string with the initial conditions given in 
Fig. 8-7. Dividing the length of the string in n parts (h = L/n), the 
initial and boundary conditions (8.58) will become 


a vo(4) foro sis” 
k = 3 
= vin ~ a(#) for= <ic<n 
n (8.61) 
Vin-1 = ¥i1 
%,; = 9 
Inj = 0 


where b is the initial height at the center of the string. Assuming the 
length of the string L = 2, the initial height b = 1/2, and dividing the 
length into 10 equal parts (n = 10), we get 


Bee 
10 


Anaand c of unity will result in 


ered (8.62) 
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or 
T=,0.2 


Using Eq. (8.60) for i= 1 to n — | will give the values of y; , for the time 
T = 0.2. Note that the value of y,, , is zero by the boundary condition. 
After these values are calculated, Eq. (8.59) can be applied for j = 1 and 
i= 1,2,3,...,n-—1 to compute the positions of the string for j = 2 or 
time = ¢ = 2 (0.2) = 0.4. The solution can proceed by calculating Yigg for 
j=3 and i=1,2,3,...,n—1and soon forj=4,5,.... 

The analytical solution of the vibrating string problem with the given 
initial and boundary conditions can be obtained from [5]. This solution 
is given in an infinite trigonometric series as follows 


yx, D0 = — b — sin — sin — cos —— (8.63) 


The numerical solution described above and the analytical solution 
(8.63) were evaluated with the aid of digital computers. The series of 
Eq. (8.63) was summed from m = 1 to 100. Table 8-3 shows the partial 
results of the calculations. An examination of this table shows that the 
values calculated by the numerical method are extremely close to the 
analytical solution. For a = 1, it can be shown that the numerical 
solution is identical to the analytical solution [6]. The slight discrepancy 
in the values as given in Table 8-3 is primarily due to the approximation 
in the evaluation on the series (8.63), that is, m to 100 instead of m = ~, 
The accuracy of the numerical solution decreases slightly as a assumes 
values smaller than one. 

For purposes of comparison, the series of Eq. (8.63) was also evaluated 
summing up to m = 5. The results are presented in Table 8-4 for time 
equal to 0.6. Note that the finite-difference solution of the table is the 
exact solution. 


SUMMARY 


The solutions given above were primarily derived for linear partial 
differential equations. However, the methods can be extended to non- 
linear partial differential equations. One such application is given in [7]. 
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Table 8-3. Results of the Solution of the Wave Equation 


ae Time = 0 Time = 0.2 
A Y-ANLYT Y-FINITE DIFF | Y-ANLYT Y-FINITE DIFF 
0 0 0 0 0 
Fe) 9.99999 E-2 4 9.99998 E-2 P| 
4 i) 2 -199999 2 
6 -299999 E) -299998 ra} 
8 -399996 4 -398986 4 
7 497974 cl 399996 4 
12 399996 4 398986 4 
14 299999 =| -299998 4) 
1.6 os =e 199999 ce 
1.8 9.99998 E-2 4 9.99998 E-2 a 
2; ~ 1.21982 E-8 0 ~ 1.30857 E-8 0 
Time = 1.0 Time = 2.0 
DIST X 
Y-ANLYT Y-FINITE DIFF 
0 0 0 0 
a -5.32606 E-10 | - 1.04774 E-9 -9.99999 E-2 =p! 
4 - 1.84689 E-9 | - 1.62981 E-9 oH =A 
6 -3.14625 E-9 | -2.09548 E-9 ~.299999 cats} 
8 -3.82612 E-9 | - 2.79397 E-9 - 399996 Si 
1 -5.67606 E-9 | -2.32831 E-9 ~.497974 =i} 
1.2 | -6.65077E-9 | -2.79397 E-9 - 399996 -4 
1.4 -8.00339 E-9 |-2.09548 E-9 =.299999 aa 
1.6 -9.46699 E-9 | -1.62981 E-9 ran "4 
1.8 | -1.08282E-8 | -1.04774 E-9 -9.99999 E-2 aad 
-2.79613 E-14 | 0 1.31982 E-8 


The stability and accuracy of the numerical solutions described above 
have been the subject of extensive study by mathematicians and engineers. 
Reference 1 contains several chapters that discuss the solution of partial 
differential equations. References 8 and 9 compare numerical and 
analytical solutions of specific partial differential equations. 


PROBLEMS 


1. Derive the expressions of Eq. (8.4). 

2. Divide Fig. 8-3 into four equal increments in each direction and 
formulate a table of coefficients corresponding to Table 8-1. Obtain 
the numerical solution and compare with the values given. 
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Table 8-4. Comparison of Solutions of the Wave Eq. (8.63) 
a 


Time = 0.6 
DIST X m= 5 | m = 100 
Y-ANLYT | Y-ANLYT | __Y-FINITE DIFF 
0 0 0 0 
2 -108262 .10099 A 
4 180754 189964 re) 
6 .205959 201149 ae 
8 201388 199561 Py? 
1. .195393 -200318 oe 
1.2 201388 -199561 + 
14 -205959 -201149 2 
1.6 .180754 189964 2 
1.8 -108262 -10099 -l 
a -1.47587 E-8 -1.17699 E-8 0 
3. Solve Prob. 2 by Liebmann’s iteration method. 
4. Derive Eq. (8.17) for the case of Laplace’s equation (8.8) for @ = 1/2. 
5. Derive Eqs. (8.20) from Eq. (8.19). 
6. Prove relations (8.22) for the set of equations 
rEg aa enw =D, 
Ao aot, + Cp, = De 
Agi B27, = D, 
7. Solve the set of equations 
-1.5 1 0 0 0 Xx; -1 
(Lee shyla 0 | [theses 0.1 
0 1 -1.5 1 0 X | = 0.1 
0 0 1 -1.5 5 X, 0.1 
0 0 0 1 -1.5 Xs =1 


by the method of Eqs. (8.22) and (8.23). 
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8. 


10. 


1g 


1 unit 


fig. P- 


12. 


ay, 


14. 
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Solve equations 


cls a ea 2m 
jasmoa Ol py 45 
pee ste eile lh llc 
a oe ee eo u 


by the method of Eqs, (8.22) and (8.23). 

Formulate the problem depicted in the example of Fig. 8-3 by the 
successive-overrelaxation method of Eq. (8.17) with » = 1.5. 
Formulate the problem depicted in Fig. 8-3 by alternating direction 
method of Eq. (8.24). Draw the logic block diagram of the solution 
for a digital computer program. 

Formulate the solution of the steady-state temperature distribution 
problem of the rectangle of Fig. P-11. 


u = 500 0 0 0 500 

1,000 1,000 

1,000 1,000 

1,000 1,000 
500 0 0 0 500 


11. 


The boundary conditions and space increments are specified. Use 
the alternating-direction method with r = 0.5. Obtain numerical 
values for one complete iteration. 

Obtain the vaiues for J = 5, t = 0.025 of Fig. 8-6 for the given heat 
transfer problem. 

Formulate the heat transfer problem depicted in Fig. 8-5, using the 
Crank-Nicholson formula (8.43). 

Formulate the heat transfer problem of Fig. 8-5, using four incre- 
ments in the X direction, by the Crank-Nicholson method of Eq. 
(8.43). Obtain numerical values for the first iteration. 


16. 
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For @ = 1/2 formulate equations resulting from the expansion of 
Eq. (8.51). 

Obtain the finite-difference solution of the wave equation for t = 0.2 
as tabulated in Table 8-3. 


REFERENCES 


1. 


Zz. 


Todd, John (ed.): “Survey of Numerical Analysis,” chap. 11, 
McGraw-Hill Book Company, New York, 1962. 

Richtmyer, R. D., and K. W. Morton: “Difference Methods for 
Initial Value Problems,” Interscience Publishers, Inc., New York, 
1967. 

BASIC Language, reference manual, General Electric Company, 
May, 1966. 

Thomas, L. H.: “Elliptic Problems in Linear Difference Equations,” 
Watson Scientific Laboratory, Columbia University, New York, 
September, 1949. 

Pipes, L. A.: “Applied Mathematics for Engineers and Physicists,” 
p. 431, McGraw-Hill Book Company, New York, 1958. 

Salvadori, M. G., and M. L. Baron: “Numerical Methods in Engi- 
neering,” p. 267, Prentice-Hall, Inc., Englewood Cliffs, N. J., 1964. 
Hovanessian, S. A., and F. J. Fayers: Linear Waterflood with 
Gravity and Capillary Effects, J. Soc. Petroleum Eng., Trans. 
AIME, March, 1961. 

Douglas, J.: “On the Numerical Integration of d2u/dx?2 + 0? u/dy = 
@u/at by Implicit Methods,” J. Soc. Ind. Appl. Math., 1955, p. 42. 


9 LINEAR AND 
SEPARABLE 
PROGRAMMING 


This discussion is intended to give the reader a basic understanding of 
linear programming and enable him to recognize and formulate linear 
programming problems. The solution of these problems can be obtained 
by the simplex method, which is discussed briefly. For a more complete 
discussion, the reader is referred to [1]. The simplex method, depending 
on the size of the problem, involves a great number of arithmetic calcu- 
lations. Therefore, the method leads naturally to solution by digital 
computers. Actually the development of digital computers has greatly 
furthered the utilization of linear programming in the solution of many 
industrial problems. Today, practically in every industrial or research 
center having access to a digital computer one can utilize digital com- 
puter codes written for the solution of linear programming problems. 

The linear programming methods have been applied to the solution of 
technical as well as economic, scheduling, and allocation problems [1]. 
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The methods have been very successfully applied in the petroleum 
industry for the solution of gasoline blending problems [1] and economic 
optimization of oil well development and exploration [2]. The linear 
programming methods, however, are limited by the fact that, as the name 
implies, they can be applied only to the solution of problems whose 
fundamental characteristics are expressed by linear equations. This 
limitation has been somewhat reduced by the emergence of a number of 
methods that extend the region of applicability of linear programming to 
restricted nonlinear regions [3]. The simplest of these methods is the 
method of separable programming [4] which will be discussed in this 
chapter. This method is an extension of linear programming which allows 
the inclusion of nonlinear constraints in the formulation of the problem. 
The method of solution of these problems is identical to the simplex 
method of linear programming. 


FORMULATION AND METHOD OF SOLUTION 


Suppose that we have a set of n nonhomogeneous equations in n 
unknowns, as follows: 


Qayx,-b, f= 1,2,...,2 (9.1) 


This set of equations has a unique solution for x 78s provided that the 
determinant formed from the a;; coefficients is nonzero. Notice that in 
the above, the number of equations and the number of unknowns are the 
same, namely, n. Now, consider the case where there are more equations 
than unknowns, i.e., 


Die’, i= 1, 23,..-07 (9.2) 


with n > m. In this case, as long as the equations of the set are inde- 
pendent, there is no solution at all to the system of equations. Finally, 
consider the case where there are more unknowns than equations: 


340 _ Digital Computer Methods in Engineering 


¥ a,;x; EA ee as Be hee er (9.3) 


with m>n. In this case there is no unique solution to the system of 
Eqs. (9.3). Introducing the additional condition 


m 
Y a;%, = minimum i=0 (9.4) 


and the restriction that the solutions should result in x; > 0 (negative 
values not accepted) may lead to a unique solution of the linear program- 
ming type or, with these restrictions, there may not be a solution to the 
system at all. In the latter case the linear programming problem is 
termed infeasible. In summary, the set of relations 


a,x; = b; ae Fe ats eer (9.5a) 
jl 
¥ a,x; = minimum i = 0 (9.5b) 
fl 
x. > 0 fj elspa (9.5c) 


with m > n, constitute a linear programming problem. Equations (9.5a) 
are called the constrains or the restraint equations while Eq. (9.5b) is the 
functional and is also referred to as the objective form. Equation (9.5c) 
is the nonnegativity requirement on the variables. Observe that the 
number of nonzero x.’s given by the linear programming solution will be 
equal to the number of constraints. 

To become familar with the terminology and the simplex method 
of solution of linear programming problems, consider the following 
numerical example: 
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8x, - Ixy < 12 (9.6a) 
—%, + Ixy 5 4 (9.6b) 
X, + Ixy 2 6 (9.6c) 
-3x, - x2 = F = minimum (9.6d) 
xy, X20 (9.6e) 


Equations (9.6a), (9.6b), and (9.6c) are the constraints, while Eq. 
(9.6d) represents the functional to be minimized. It will be seen later 
that the inequality constraints (9.6a)-(9.6c) are converted to equality 
constraints by addition of slack variables. Addition of these variables 
will increase the number of unknowns and render the problem into the 
form where the number of the unknowns is greater than the number of 
equations. (If the functional is to be maximized, the coefficients of the 
variables in the functional are multiplied by —1. Then the minimization 
of the resulting functional is equivalent to maximizing the original 
functional.) Equation (9.6e) gives the nonnegativity requirement on the 
variables x, and x,. Note that all of the constraints have positive 
right-hand sides. The above constraints and the functional are represented 
graphically in Fig. 9-1. In this figure, the constraint 3x, - 2x, < 12 
represents the region to the left of the line AA, the constraint 
~x, + 2xy S 4 represents the region below the line BB, and the constraint 
x, + 2x, 2 6 represents the region above the line CC. Thus, the region 
covered by small squares represents the allowable region for the solution 
of the problem. The dashed lines are for various values of the functional 
F. From this figure, the constrained minimum of the function F occurs 
at x, = 8 and xo = 6 and has a value of F = —30. Now let us solve the 
same problem by the simplex method. 

First, replace the inequality constraints (9.6a), (9.6b), and (9.6c) by 
equality constraints. This is accomplished by adding slack variables 
Xg, X4. X5, and xg to the system, as follows: 


3x, - 2xp + xg eng (9.7a) 


=X, + 2x +X, =4 (9.7b) 


f] 
ior} 


X1 + 2X9 +X5 — Xe (9.7¢) 
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© x1'+ 2x2 >6 


F = -3xy —x2 = min 


figure 9-1. Simplex method for the example problem. 
-3x,) - xX = F (9.7d) 
Ky, Mgr! gw yy I Xpe KGS 0 (9.7e) 


Note that Eqs. (9.6a) and (9.6b) are “smaller than” inequalities and they 
can be changed to equalities (9.7a) and (9.7b) by adding positive slack 
variables x, and x4. On the other hand, Eq. (9.6c) is a “greater than” 
inequality and it can be changed to an equality by adding a slack 
variable x, with a —1 coefficient. But this, as will be seen later, is not 
computationally desirable. Therefore, add another slack variable x; 
with a +1 coefficient, At this point the constraint (9.6c), represented by 
equality (9.7c), will be satisfied only when x; = 0 and xg 2 0. Thus, as 
long as x; 2 0 we will not be in the allowable region of Fig. 9-1 and the 
solutions will be somewhat artificial.* Equation (9.7d), as before, repre- 
sents the functional to be minimized, and Eq. (9.7e) is the nonnegativity 
requirement on all of the variables. 


*Slack variables performing functions similar to that of Xs are sometimes called 
artificial slacks. In some problems, in order to derive these variables out of the final 
solution, they are included in the functional with arbitrarily large coefficients 
(penalty coefficients). 
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Start the simplex method by taking the solution 
x%,=0 x,=0 x,= 12 x,=4 x,= 6 xg, = 0 


which makes F = 0 for the first iteration. Observe that this solution 
satisfies the constraints ((9.7a), (9.7b), and (9.7c)) and the nonnegativity 
requirement (9.7e). In linear programming, solutions of this type, having 
the same number of nonzero variables as there are constraints, are called 
basic solutions, and the nonzero variables are said to be in basis. At this 
point the question is posed—can the functional assume a lower value? 
The answer is yes, since, in the functional (9.7d), x; and x, have negative 
coefficients. Therefore, increasing the value of either one from zero 
(given by iteration 1) will decrease the value of the functional. In 
particular, since x, has a larger negative coefficient, a higher rate of 
decrease of the functional can be obtained by increasing x;. The amount 
that x, should increase can be obtained from the following observations. 
Increasing the value of x, in a particular constraint will result in an 
increase, a decrease, or no change in the values of the other variables 
present in the constraint. We need not concern ourselves with variables 
which increase or remain unchanged, but should be careful that de- 
creasing variables do not become negative. This might arise only in 
constraints in which x, appears with a positive coefficient. Thus, we 
only examine constraints (9.7a) and (9.7c) of the problem. From Eq. 
(9.7a), bringing x, in basis will decrease the value of x; (which is in 
basis in iteration 1). The highest value that x, can have (with x3= 0) is 
12/3 = 4. Similarly, from Eq. (9.7c) x, can attain a value of 6 with 
x; = 0. Thus, take the smaller of these values, 4, and bring x, in basis 
through the constraint (9.7a). Dividing Eq. (9.7a) by the coefficient of 
x,, We obtain the set of equations 


x, - 2x» + x3 =: 4 (9.8a) 
=X, + 2k 4 =4 (9.8b) 
X41 + 2x» + X53 - Xg = 6 (9.8c) 
-8x,; - Xp =F (9.8d) 
Sy.” Xa Kg Xgr Xe, Xe = 0 (9.8e) 
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Eliminating x, from Eqs. (9.8b), (9.8c), and (9.8d) by multiplying Eq. 
(9.8a) by proper constants and adding to Eqs. (9.8b), (9.8c), and (9.8d) 
results in 


xy - 2x, + ix, = 4 (9.9a) 
xy + 3x3 + Xq =8 (9.9b) 
8x. - ixg PS ee ee (9.9¢) 
- 8X9 + X3 = F+ 12 (9.9d) 


The basic solution of this iteration is 


This results in 


F+12=0 
or 


F = -12 


Thus, in this iteration the value of the functional F is decreased by 12 
units, Further decrease of the functional can be obtained by increasing 
Xy, which has a negative coefficient in Eq. (9.9d). Consider constraints 
(9.9b) and (9.9c), in which x5 appears with a positive coefficient. From 
Eq. (9.9b) we note that x, can attain a limiting value of (8)/(4/3) = 6 
and from Eq. (9.9c) a limiting value of (2)/(8/3) = 3/4. Thus, we bring 
Xg in basis through constraint (9.9c). Dividing Eq. (9.9c) by 8/3, we have 


xy - 2x» + 3X3 = 4 (9.10a) 
xp + 3X3 + Xq = 8 (9.10b) 
Xq - dxg + Bxs - xg ees 3 (9.10c) 


-3x%9 + Xg = Fx 12 (9.10d) 
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Eliminating x, from Eqs. (9.10a), (9.10b), and (9.10d), we get 


xy 4x3 + Bs - ix¢ = 2 (9.1 1a) 
3x3 +k = $%5 + ixg =7 (9.11b) 
1 
X_ - 3x, + 3x, - 3x, = 3 (9.11) 
5 9 9 
3*3 + 3%5 — g%6 = F+ a (9.11d) 
The basic solution of the above iteration is 
x,-% x, - 3 %ia=0 x,=7 x= 0 x= 0 
This solution gives 


BeOS ne) 1A D5 
4 


Observe that in this iteration x, is zero and the solution is no longer 
artificial. We refer to this solution as a basic feasible solution. From 
Eq. (9.1 1d) we observe that the functional F can be further decreased by 
increasing xg, which appears with a negative coefficient. Equation (9.11b) 
gives the largest value that x, can have, that is, (7)/(1/2) = 14. Thus we 
bring x, in basis through Eq. (9.11b) by eliminating x, in Eqs. (9.11a), 
(9.11c), and (9.11d). This results: 


os + ix, & 4x, = 8 (9.12a) 
Xj + 2x4 -%,+%g = 14 (9.12b) 

kp + ley + 3x, -6 (9.12¢) 

12, ts 2x4 = F+30 (9.12d) 


The basic feasible solution of this iteration is 


%=8 x =6 x,=0 x4, =0 x, =0 xg = 14 
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This solution results in a functional F = —30. Since negative coefficients 
do not appear in Eq. (9.12d), the basic feasible solution 


with a functional of F =—30 is the optimal solution. 

The iterations together with the corresponding functional values are 
expressed in Fig. 9-1. The first iteration with x, = 0 and x. = 0, F = Ois 
at point (1); the second iteration with x, = 4, x. = 0, F =—12 is at point 
(2); and so on. Observe that each iteration moves the solution from one 
corner to the other, while decreasing the value of the functional. Note 
that, since at iteration 3 the solution becomes feasible, point (3) is a point 
of the allowable region. 

The simplex method, as described above, and its variations are used for 
the solution of linear programming problems. The simplex method can 
easily be coded for digital computers. The rules to be followed in the 
solution of linear programming problems by the simplex method can be 
summarized as follows: 

1. Find the x, with the highest negative coefficient in the functional. 

2. Locate the constraints in which x; has positive coefficients. 

3. Divide the right-hand side of these constraints by the respective 

x; coefficients, 

4. Choose the smallest of the above values and the respective con- 

straints. 

5. Divide this constraint by the coefficient of x; and eliminate x; 

from all of the other constraints and the functional. 

6. Repeat the above procedure. 

The discussion of the linear programming method has thus far covered 
only the basic principles of the method. The theory and the computa- 
tional aspects of linear programming have developed considerably beyond 
the presentation of this chapter. The interested reader is referred to 
{5, 6] for further study of the subject. 


SEPARABLE PROGRAMMING 


Separable programming is a generalization of the simplex method of 
linear programming for certain types of nonlinear problems. This 
generalization of the method was developed by C. E. Miller and presented 
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In the above equations n,’s are parameters. The first two equations 
((9.13a) and (9.13b)) give the values of x and y asa function of param- 
eters n;. Equation (9.13c) together with the requirement that no more 
than two of the ns can be nonzero and these must be consecutive will 
restrict the x and y values to the line segments. The meaning of this will 
become clear when we express Eqs. (9.13) numerically for y = f(x) of 
Fig. 9-2. Reading the values of x;and y,; from the figure, we have: 


x = n,() re ny (2) me n,(4) + n,(6) cf n;(9) + ng (10) (9.14a) 
y = n, (1) + ng (4) + n3(6) + n4(7) + n5(6) + ng (5) (9.14b) 
1 = nj) + ng + ng + Ng t+ Ns + Ng (9.14c) 
My, Mgy Ngy Mg, M5, Ng 2 0 (9.14d) 


For example, a solution satisfying Eqs. (9.14c) and (9.14d) and the 
consecutive n; requirement will be n; = 0.5, and ny = 0.5 with 
Nn, = Ng = Ns = ng =O. This gives 


x = 0.5(4) + 0.5(6) 5 


6.5 


(9.15 
y = 0.5(6) + 0.5(7) / 


This is, as expected, a point on the line segment v,v,. Another solu- 
tion, for example, satisfying Eqs. (9.14c), (9.14d), and the consecutive 
n, requirement will be n, = 1.0 with the rest of n;’s equal to zero. This 
corresponds to the point (4, 6) of Fig. 9-2, which is the end point of a 
line segment. We emphasize again that in the linear programming formu- 
lation n,’s represent a set of parameters that are treated as unknowns. 
Note that the condition specified by Eq. (9.14d) need not be incorporated 
in the formulation, since the method of linear programming deals only 
with positive or zero values of the unknowns. 

The above method can be extended to three or more dimensions. 
Three-dimensional relationships, geometrically expressed by surfaces, can 
be replaced by a number of triangular planes approximating the surface. 
For example, the surface f(x,y, z) = 0 can be approximated by a number 
of triangular planes, and equations corresponding to Eqs. (9.13) can be 
written: 
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xe das (9.16a) 
y = 2 ni¥; (9.16b) 
z= ; 1,2; (9.16c) 
ie > n, (9.164) 
n, 20 (9.16e) 


In the above equations, x,, y;, and z; represent the coordinates of the 
vertices of triangular planes, andn,;’s are parameters. Together with the 
condition that no more than three (as compared with two of the previous 
case) of the n,’s can be nonzero and these must be associated with a 
triangular plane approximating the surface, Eq. (9.16d) will restrict the 
x, y, and z values given by Eqs. (9.16a), (9.16b), and (9.16c) to the 
triangular plane. This becomes clear by taking the example of a surface 
approximated by only one triangular plane with vertices (1, 1, 1), 
(2, 2, 3), and (1, 2, 2), writing the equation of the plane passing through 
these, and showing that the points obtained from Eqs. (9.16) are located 
on this plane. 

The general equation of a plane is written 


Ax + By + Cz = 1 (9.17) 


Substituting the x, y, and z values at the vertices, we obtain a set of three 
equations and three unknowns, as follows: 


A+ B+ C 


" 
e 


9.18 
2A + 2B + 3C = 1 ¢ ) 


A+ 2B + 2C 


il 
= 
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Solving the above equations for A, B, and C(A = 1, B = 1, C = -1) 
and substituting in Eq. (9.17), we obtain the equation of the plane 
passing through the vertices 


x+y-ze=1 (9.19) 


On the other hand, Eqs. (9.16) can be written for the triangular plane 
with the specified vertices as follows: 


® 
it 


ny) + ng(2) + ng 


< 
" 


ny (D) + ng(2) + ng (2) 
(9.20) 


n 
fT 


n, (1) + ng(3) + ng (2) 
1 = nj +g + ng 
Ny, Ng, Nz 2 0 


For example, the values of n, = 0.4, ng = 0.3, and ng = 0.3 will 
satisfy the last two equations as well as being then,’s associated with the 
triangular surface. The corresponding values of x, y, and z obtained from 
Eqs. (9.20) should be on the plane described by Eq. (9.19). Substituting 
the specified n;’s in Eqs. (9.20), we have 


oe L8 
y = 1.6 
z= 1.9 


which, indeed, satisfy Eq. (9.19) and hence are on the triangular plane. 
The above principles have been used extensively in the formulation of 
the following problem. 


FORMULATION OF A NUMERICAL PROBLEM 


The formulation of separable programming problems is illustrated by 
two numerical examples. The problems selected are to find the optimum 
operating conditions in a power plant with three generators each with 
different power-fuel consumption characteristics [7]. 
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Case 1 


The first case is that of a power plant with three boiler-turbine- 
generator combinations with nonlinear generated power vs. fuel consump- 
tion characteristics. Two of the boilers use fuel oil and the third uses 
coal. The power plant load, combined outputs of the three generators, 
and the fuel costs are specified. The objective is to find the operating 
level of each generator to minimize total fuel costs. 

The generated power-fuel consumption characteristics are given in 
Fig. 9-3. The i,’s, y;’s, and v,’s specify points on the curves of the 
boiler-turbine-generator combinations A, », and v, respectively. The 
nonlinear curves are replaced by piecewise linear functions shown in the 
figure. For example, the curve of the boiler-turbine-generator combi- 
nation v is replaced by linear segments vj v9, vgvg,vgvq, and vgv5. 
Using these segments, the following equations for the generated power, 
fuel consumption, and the sum of n,’s can be written: 


25n, + 23ng + 19.5ng + 16.7n4 + 13n, = G, (9.21a) 
8.40n, + 7.86ny + 6.99ng + 6.45n, + 5.87n, = F, (9.21b) 
Ny + Ng + Ng t+ Ny + Ns = 1.0 (9.21c) 


where G, and F,, are the generated power and coal consumption of the 
boiler-turbine-generator combination v. Note that the last equation, 
(9.21c), along with the additional requirement that only two consecutive 
n;'s are to be nonzero will guarantee a solution on one of the specified 
line segments, as discussed previously. In the cases examined, the 
consecutive requirement was automatically satisfied, mainly because the 
generated power-fuel consumption curves were concave upward,* as 
noted by the following two observations. First, consider generator v 
of Fig. 9-3 operating at 18 MW; the corresponding fuel consumption on 
line v3v4 is lower than on any other line passing through two v’s, for 
example, vgv5. Second, consider the main objective of the formulation 


*This argument does not apply where the generated power-fuel consumption 
curves are concave downward—a condition rarely encountered in power plant opera- 
tions. In these cases the additional provisions (specified in [4] ) dealing with consecu- 
tive nonzero n; and m; values should be incorporated in the linear programming code. 
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figure 9-3. Generated power vs fuel consumption for Case 1. 


which is to minimize the amount of fuel (within the specified constraints). 
From these it is easy to see that at any operating level, the solution will 
seek the lowest obtainable fuel consumption, thereby satisfying the 
consecutive n; requirement. 

Similar equations can be written for the other two boiler-turbine- 
generator combinations \ and ». The parameters associated with these 
are ] and m, respectively. Note also that these use fuel oil as the boiler 
fuel. 

After completing these calculations, the linear programming matrix of 
the problem can be constructed as given in Fig. 9-4. The matrix contains 
a listing of the unknowns, the constraint equations, and the functional, 
the latter representing the objective function to be minimized. The right- 
hand side (RHS) of the equations is given on the extreme right of the 
matrix. The linear programming matrix of Fig. 9-4 was solved by a 
computer code using the simplex method of solution. 

In the matrix, G’s and F’s refer to generated electrical power and fuel 
consumption, respectively. The /’s, m’s, and n’s specify parameters 
associated with the linearized curves of Fig. 9-3. Constraints 1 to 3 are 
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for boiler-turbine-generator combination A, 4 to 6 for p, and 6 to 9 for v. 
The constraint equation of row 10, 


G,+G,+G, = 54.0 MW (9.22) 


specifies the total load on the power plant. The functional represents the 
cost of fuel consumed by the plant. It is assumed that fuel oil costs 
12.6 $/ton and that coal cost 10 $/ton. Thus, the functional becomes 


12.6F, + 12.6F,, + 10F, = minimum (9.23) 


as given on the last row of the matrix. 

The solution of the linear programming matrix gave the numerical 
answers shown on Fig. 9-4. The corresponding operating levels are 
specified by ©) on Fig. 9-3. Note that generators A, », and v operate 
at 9, 21.9, and 23 MW, respectively. The total fuel cost (functional) 
is 169.5 $/hr. 


Case 2 


The second case is that of a power plant with two boiler-turbine- 
generator combinations, each capable of using a mixture of two types of 
fuel oil. The fuel consumption-load characteristics are nonlinear, while 
the fuel consumption-type of fuel characteristics are a linear function of 
the fuel blending ratios. The total electrical output of the power plant 
and the amount and type of fuel available at zero cost are specified. The 
object is to minimize the amount of purchased fuel. 

The generated power-fuel consumption characteristics of this case are 
given in Fig. 9-5. Note that they can be considered as surface with fuel 
constituting the third dimension. The boiler-turbine-generator combi- 
nations A and » represent systems the boilers of which can use type-A or 
type-B fuel oil or any combination of these. For example, \ operating at 
20 MW will use 5.5 tons per hour of type-B8 fuel oil or 5.65 tons per hour 
of a 1:1 ratio of type-A and type-B fuel oil. Connecting three consecutive 
d’s or p’s, we will obtain triangular planes for each of the two sets of 
surfaces. Thus, in the present case, we can have three nonzero values of 
1, Or m; as opposed to two nonzero values in the previous cases. We 
should be careful, however, that the nonzero /; and m; values in the final 
solution represent a plane which is the closest approximation to the 
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figure 9-5. Generated power vs fuel consumption for Case 2. 


generated power-fuel consumption surface passing through the same /;’s 
or m,’s. From the results of the cases examined, because of the concave 
upward nature of the surfaces this condition was automatically satisfied. 

After obtaining the coefficients from Fig. 9-5 we can set up the 
linear programming matrix of Fig. 9-6. The last two constraints specify 
the amount of available fuel at zero cost (10 tons of type-B fuel) and the 
desired power plant output (50.0 MW). The objective is to minimize the 
amount of purchased type-A fuel oil. F,, and F'4,, Tepresent type-A fuel 
used in generator-turbine-boiler combinations A and p, respectively. 
Similarly, F,, and Fp, Tepresent type-B fuel used in generator-turbine- 
boiler combinations \ and pz. 

The solution gave the results reported in Fig. 9-6 and the corresponding 
operating points specified (©) in Fig. 9-5. Note that generators A and p 
operate at approximately 23.5 and 26.5 MW, both below their rated 
capacities. The total type-A fuel oil consumption was 3.17 tons/hr. 
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PROBLEMS 


i 


Obtain the solution of the following linear programming problem by 
the simplex method: 


2%, - % <8 
=x, + 3x, < 6 
X1 + 4% = maximum 


Show that the solution occurs at the intersection of the lines 


2x, - X_ = 8 


Answer: x, = 6, x5 = 4. 


i 
a 


=a 3x5 = 


Obtain the solution of the linear programming problem 


Answer: x, = 4.235, xX» = 4.117. 


x) + X= maximum 


Write the equation of a plane passing through (1, 1, 1), (3, 4, 6), and 
(2, 3, 1) and equations corresponding to Eq. (9.20) of the text. 
Show that the point n, = 0.2, ny = 0.2, and ng = 0.6 is on the plane. 
Formulate the separable programming matrix for the case of a 
power plant with two generator-turbine-boiler combinations A and 
» with the following characteristics 


Fuel consumption, Generated power, 
tons/hour megawatts 
6 10 
Generator A 7 15 


10 20 
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Fuel consumption, Generated power, 
tons/hour megawatts 
5 10 
Generator / 6 15 
10 20 


It is required to generate 35 megawatts, minimizing fuel consump- 
tion. 
5. Obtain the solution of the linear programming problem 


X1 — X%_ + 2xg- x4 = 2 Answers: X, = 8, x9 = 


fl 
oo 


2x, + X%q - 8x4 + X4 = 
Xp +X + Xgtxy= 7 


-2x,-X9 + X3 + xX4 = maximum 


6. Obtain the solution of linear programming problem 


2x, + X, 2 2 Answers: x, = 3 
xX, = 0 
xX, + 8x_ S38 
< 
2x, 54 
3x, - X9 = maximum 


7. Write the logic for a digital computer program for the simplex 
method, 
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70 DYNAMIC 


PROGRAMMING 


Dynamic programming has been successfully applied to optimizing 
returns from a set of engineering units, operating in series or in parallel, 
to produce certain outputs. In chemical engineering applications, the 
chemical process of production can be divided into a series of stages 
(units) where the output of one stage constitutes the input of the 
next stage. In industrial engineering applications, the productian process 
can be considered to be accomplished in a series of units each requiring 
a certain amount of effort for a specified return. The problem of finding 
the most economical operating levels of electrical generators (each with 
different efficiency-load characteristics) in a power plant can be formu- 
lated by dynamic programming. 

This discussion is intended to give the reader an understanding of the 
basic principles of dynamic programming. Numerical examples of parallel 
and series operation of production units are given. The results obtained 
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for the parallel case are compared with the solutions obtained by the 
method of separable programming described in the previous chapter. 
GENERAL DYNAMIC PROGRAMMING ANALYSIS 


Consider a certain engineering production operation consisting of n 
units, as shown in Fig. 10-1. For convenience in calculation, number the 


Uniti 
nt vi 
‘decision}(output) 


(xj-1) 


(return) 


Total return R 
figure 10-1. Dynamic programming configuration of production units. 


last unit (on the right) as unit 1 and the first unit on the left as unit n, 
and assume the following notation: 


input to unit 7 

output of unit 7 

decision required at unit 7 
‘ return of unit i 


nou 


x 
y 
d 
R 


In the above notation, x;, y;, and d; can represent vectors. This means 
that input and output may consist of several components and there can 
be several types of decisions possible at each unit. Using the principle of 
dynamic programming, the total return R can be optimized (maximized 
or minimized) as follows. 

Starting with unit 1, optimize the return of this unit R, with respect 
to its input x,, output y,, and decision d,. Select the optimum operating 
points of unit 1 and combine this with operating points of unit 2. 
Optimize the combined returns of units 1 and 2. Select the corresponding 
optimum operating points. Combine these points with operating points 
of unit 3 and calculate the corresponding returns. Select the optimum 
operating points of unit 3 and the corresponding returns. Continue this 
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process. This procedure is governed by the “principle of optimality” of 
dynamic programming [1]: 


No matter what the input (x;) and decision (d;) and resulting 
output (y) at unit 7 may be, the succeeding decision, d; , v 
must be made in such a way as to yield the optimum return 
from the i + 1 units. 


Let us formulate the mathematical expression of the above principle for 
the serial multistage process of Fig. 10-1. Let us further assume that the 
input to each stage is equal to the output of the previous stage, i.e., 


Se test (10.1) 


For general stage i of the n-stage system of Fig. 10-1, we can write the 
expressions for the output and the return as a function of the input to 
stage i and the decision made at stage i. 


X;_1 = b;(x;,d) (10.2) 


1, = r(x; d;) 


where x;_, is the output of stage i (the input of stage i— 1) and ¢; and r; 
represent functions of variables in parentheses. Since the output of every 
stage could be expressed as a function of its input and the stage decision, 
starting from stage n and proceeding to n— 1, n— 2,.. . and finally to 
i+ 1 which supplies the input of stage i, we can write 

Deg tet hey (Gee Ad,» Shy gy) (10.3) 
which represents the input of stage i as a function of system input x, and 
the decisions made up to stage i. Using this expression in the return 
(Eq. (10.2)), we can express 


r= 1%.) 4), --+,d) (10.4) 


In comparing Eqs. (10.3) and (10.4), we see that the decision of stage 
i,d,, affects only the return of this stage and, of course, later stages. 
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The total n-stage system return R will be a function of individual 
returns from each stage; using Eq. (10.4), the total return can be 
expressed 

R= gir, %,» d,), r 


AS eth hy cnn Fe, 8 +, d,)) 


m=1 aie 


(10.5) 


The optimization problem will require the maximization or the minimi- 
zation of the above return as a function of system input x, and stage 
decisions d,,d,_,,.--,4,. Using Eq. (10.2) for stage returns, the total 
return can be expressed also as 


ReMANO he (6 558) dy on alg ys Oy) (10.6) 


=n 
Consider maximization of return R where 


R= 1 (%p9 4) + Ty 4% 7d yt eee + 17 (x,, dy) (10.7) 


uo mat Ratt 


represents the sum of individual stage returns. Using the optimality 
principle and denoting by R, the optimum i-stage return (stages 1 through 
i), we write 


i 


R, = maxtr, (py d) + R;_4] (10.8) 


The above is the recursion equation of dynamic programming. Starting 
from i = 1 and proceeding toi = 2, 3,...n the total return of i stages are 
maximized as a function of stage i decision d;. Thus, the total n-stage 
system maximization problem is reduced to maximizing returns from 
1, 2, 3,... , stages, respectively, and combining the cumulative return 
of (i — 1) stages with the ith stage, and so on. As will be seen in the 
following examples, use of the methods of dynamic programming greatly 
reduces the number of calculations required in a particular optimization 
over the number which would be required if one had to choose the 
optimum from all of possible combinations of decisions. 


NUMERICAL EXAMPLES 


The following numerical examples will help to clarify the above 
principle. 
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Example of Parallel Operation 


Consider a power plant with three boiler-turbine-generator combina- 
tions A, B, and C, with the cost vs. load curves given in Fig. 10-2. From 
this figure the values of cost for various values of generated power, for 


Operating cost, dollars /hr 


Operating pts. for 60 MW: 


10 15 20 25 30 


Generated power, MW 


figure 10-2. Operating cost vs. load for generators A, B, and C. 
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each of the three generators, can be obtained and tabulated. This 
tabulation for increments of 5 MW is shown in Table 10-1. Suppose that 
we want to generate a total of 60 MW in the most economical way and 
that we proceed to formulate the problem using dynamic programming. 


Table 10-1. Generated Power and Cost for Generators A, B, and C 


Generated Cost, $/hr 
power, MW 


Generator C 
Generated Cost, 
power, MW | $/hr 


Generator A 
Generated Cost, $/hr 
power, MW 


10 30 10 40 10 45 
15 40 15 46 15 47 
20 51 20 55 20 51 
25 66 25 65 25 


30 90 30 78 30 


Let us take generator A operating at 10 MW and B operating at 10, 
15, 20, 25, and 30 MW, respectively, and tabulate the values given in 
set 1 of Table 10-2. The first row of this set gives the cost of generating 
20 MW, 10 by A and 10 by B, as 70 $/hr, which is the sum of cost 
figures from Table 10-1 (30 + 40 = 70 $/hr). Ina similar manner, A 
operating at 10 MW and B at 15 MW will cost, from Table 10-1, 30 + 46 
= 76 $/hr, as given in the second row of the first set in Table 10-2. 
Hence, we can complete the rest of the values of Table 10-2 with the aid 
of Table 10-1. Table 10-2 has, in fact, all of the possible operating 
combinations of generators A and B. 

Up to this point we have merely tabulated values. At this time we 
utilize the governing principle of dynamic programming to reduce the 
number of computations when adding the output and costs of generator 
C to the respective values of generators A plus B as given in Table 10-2. 
An examination of Table 10-2 shows that there are several combinations 
of generators A and B that will result in the same amount of total power. 
For example, we can generate 35 MW by operating A at 10 and B at 25, 
as given in the fourth row of set 1; or A at 15 and B at 20, as given in the 
third row of set 2; or A at 20 and B at 15, as given in the second row of 
set 3; or A at 25 and B at 10, as given in the first row of set 4. But 
from all of the mentioned combinations for generating 35 MW, the 
combination of A at 10 and B at 25 with a total cost of 95 $/hr, as 
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Table 10-2. Operating Combinations of Generators A and B 


Set A operating B operating Cost(A + B) | Total power 
at MW at MW $/hr (A + B),MW 
10 10 70 20* 
10 15 76 25* 
1 10 20 85 30* 
10 25 95 35% 
10 30 108 40 
15 10 80 25 
15 15 86 30 
2 15 20 95 eh 
15 25 105 40* 
15 30 118 45 
20 10 91 30 
20 15 97 35 
3 20 20 106 40 
20 25 116 45* 
20 30 129 50* 
25 10 106 35 
25 15 112 40 
4 25 20 121 45 
25 25 131 50 
25 30 144 Bk 
30 10 130 40 
30 15 136 45 
5 30 20 145 50 
30 25 155 55 
30 30 168 60* 


given in the fourth row of set 1, is the most economical.* In a similar 
manner we can pick up the most economical ways of generating 20, 
25,..., 60 MW from Table 10-2 as marked by asterisks. From now on 
we will be concerned only with the most economical combinations and 
costs of operating generators A and B and we can, in effect, drop all of 
the other values. Thus, in incorporating the generated power of generator 
C we use the combinations noted by asterisks in Table 10-2. These values 
are listed in Table 10-3. In this table we have two additional columns—one 


*Note that 35 MW can also be generated at 95 $/hr by operating A at 15 andB 
at 20 as given in the third row of set 2. In case of ties either combination can be 
used for further study. 
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Table 10-3. Operating Combinations of Generators A + B and C 


Generated power Cost(A + B) C,MW 

(A + B),MW $/hr 60- (A + B) cost 
20 70 
25 76 
30 85 30 152* 
35 95 25 154 
40 105 20 156 
45 116 15 163 
50 129 10 174 
55 144 
60 168 


for the load on generator C and the other for total cost. Since the total 
generated power as given in the problem is to be 60 MW, we calculate 
the load of generator C on this basis. For example, when A + B are 
generating 20 MW, C should be generating 60 — 20 = 40 MW. But this, 
from Fig. 10-1, is outside the operating range of C, so that no numbers 
are entered in the first row of the third and fourth columns, When 
A + Bare generating 40 MW, C will be generating 60 — 40 = 20 MW; the 
cost will be 105 + 51 = 156 $/hr, where 51 $/hr is the cost of C operating 
at 20 MW. 

The rest of the values of Table 10-3 can be computed in the same 
manner. From the total cost column of this table it is seen that the 
152 $/hr gives the most economical way of operating the generators for 
generating 60 MW of power. We see that the most economical operation 
corresponds to C operating at 30 MW and A + Bat 30 MW. Returning 
to Table 10-2, we obtain the operating levels of generators A and B which 
give the total of 30 MW. This value is given on the third row of set 1, 
and corresponds to A operating at 10 MW and B operating at 20 MW. 
These points (A at 10, B at 20, and C at 30) are marked on Fig. 10-1. 

Now let us ask ourselves what we have gained by using dynamic 
programming. First, if we were to do this problem by considering all 
possible operating combinations of generators A, B, and C and correspond- 
ing costs, we would have 5 x 5 x 5 = 125 combinations to consider (each 
generator can operate at 5 levels, that is, 10, 15, 20, 25, and 30). With 
dynamic programming we have reduced the number of calculations to 
25 (in Table 10-2) + 5 (in Table 10-3) = 30. This is a reduction of over 
75 percent. Second, we have followed a logical pattern for performing 
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computations that can easily be programmed for a digital computer. The 
saving in computation will become more evident if we consider a finer 
grid for operating ranges of generators. For example, if we consider an 
increment of 2 instead of 5 MW we will have 11 load values for each 
generator, that is, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, and 30 MW. The 
total number of possible operating combinations will be 11 x 11 x 11 = 
1,331. By dynamic programming the number of combinations will be 
reduced to 11 x 11 + 11 = 132 —a reduction of over 90 percent. 

The solution given by Table 10-3 not only gives the most economical 
operating levels, but also gives other operating levels with increased cost. 
Sometimes it is desirable to operate at a level that is more practical than 
economical. For example, if we do not want to operate any of the 
generators at their rated capacity (30 MW), we might consider the solution 
corresponding to the total cost of 154 $/hr as given in Table 10-3. This 
solution results in operating A at 10, B at 25, and C at 25 MW; but of 
course this will cost us 154 — 152 = 2 $/hr more than the optimum 
solution. 

Now we ask ourselves whether if we had considered generators C and 
B first and then added generator A we would still get the same answers. 
The answer is that we will get the same optimum answer (A at 10, 
B at 20, C at 30 with a cost of 152 $/hr), but the other combinations 
will not be the same. This can be verified by taking generators C and B 
and completing a table similar to Table 10-2 and then incorporating 
generator A and completing a table similar to Table 10-3. This was done, 
and the final table of computations is shown in Table 10-4. A comparison 
of Tables 10-3 and 10-4 shows that optimum operating combinations 


Table 10-4, Operating Combinations of Generators C + B and A 


Generated power Cost (C + B) A, MW Total 
(C + B), MW $/hr 60 — (C + B) cost 
20 (C at 10, B at 10) 85 
25 (Cat 15, B at 10) 87 
30 (C at 20, B at 10) 91 30 181 
35 (C at 20, B at 15) 97 25 163 
40 (C at 25, B at 15) 105 20 156 
45 (C at 30, B at 15) 113 15 153 
50 (C at 30, B at 20) 122 10 1s2* 
55 (C at 30, B at 25) 132 
60 (C at 30, B at 30) 145 


—_——— 
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and corresponding costs are the same in the two tables, but the inter- 
mediate operating combinations and costs are not the same. For example, 
from Table 10-4, the next lowest cost to the optimum is 153 $/hr while 
in Table 10-3 the corresponding value is 154 $/hr. 

We solved the load allocation problem and presented the results in 
Table 10-3 for the total power generation of 60 MW. Using the first two 
columns of Table 10-3 we can find the most economical generating levels 
of the three generators for any other value of total generated power. The 
results of these calculations are illustrated in Fig. 10-3. From this 
figure, for example, we observe that the most economical ways of 
generating 75 MW and 80 MW are operating A at 20, B at 25, and Cat 
30 at a cost of 182.5 $/hr and operating A at 20, B at 30, and C at 30 at 
a cost of 195 S/hr, respectively. This figure can also be used for obtain- 
ing operating levels that are not multiples of 5. For example, if we want 
to generate 78 MW at minimum cost, this value can be obtained from 
Fig. 10-3 by operating A at 20, B at 30, and increasing the operating 
level of C to 25 + 3 = 28 MW. The cost, by linear extrapolation, will 


250 


225 


75 


150 


Minimum cost, dollars /hr 


125 


100 


30 40 50 60 70 80 90 100 
Total generated power, MW 


figure 10-3. Most economical power generation levels of generators A, B, and C 
for required total power generation. 
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be approximately 182.5 + 3/5 (192 — 182.5) = 190 $/hr. Of course, a 
more accurate answer can be obtained by solving the problem with 
load increments of 1 MW. 

To compare the above solutions with the solutions obtained by other 
methods, we will solve the problem by the method of separable program- 
ming. 


Separable Programming Formulation 


The problem described above can also be solved by separable program- 
ming, as discussed in the previous chapter and in [2]. In formulating the 
problem for this method we need to substitute piecewise linear approxi- 
mations for the nonlinear cost-load curves of Fig. 10-2. This approxi- 
mation is shown by dashed lines in the figure. The equations for 
generated power and cost can be written in terms of “special variables” 
which incorporate the piecewise linear approximations of the nonlinear 
curves. The equations for generator A can be written as follows: 


-P, + 10d, + 17.5Ag + 26.2503 + 30A4 = 0 
-Ca + 30d, + 45r, + T0Ag + 90AY = 0 (10.9) 


Ay tAgtAg t+ Aq = 1 


where Aj, A, Az, and A, are the special variables and their coefficients in 
the first two equations are the coordinates of the intersections of line 
segments describing the curve of generator A in Fig. 10-2. The first two 
equations give the amount and cost of generated power, respectively. 
The last equation, with the additional condition that only two of the ’s 
can be nonzero and that these must be consecutive, will guarantee a 
solution on one of the specified line segments. Equations similar to 
Eq. (10.9) can be written for generators B (with special variables ) and 
C (with special variables y). The separable programming matrix of the 
problem is shown in Fig. 10-4. Rows 1 through 9 of the matrix give 
constraint equations for generators A, B, and C, while row 10 specifies 
the total required amount of generated power, i.e., 


Bie peee ae t60 (10.10) 


In Fig. 10-4 the designation RHS specifies right-hand side of equations. 
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The solution of the problem by the simplex method of linear program- 
ming is shown at the bottom of the figure. This solution results: 


Generated power, MW Cost, $/hr 
P,= 10 30 
. Pg=20 55 
Po= 30 67 
Total 60 152 


Note that the above solution is exactly the same as obtained by dynamic 
programming. 

The method of dynamic programming as applied to the above problem 
is simple to follow and the calculations involved are logical and can be 
performed by digital computers with a small amount of programming 
effort. The accuracy of the solution can be improved, if necessary, by 
taking smaller increments for the generated power. This will increase the 
number of calculations to be performed although the logical sequence of 
calculations will remains the same. 

The solution obtained by separable programming will depend on the 
accuracy of representation of cost-load curves by piecewise linear seg- 
ments. We note that this accuracy can be easily increased by taking more 
linear segments to represent a given curve. This in turn will increase 
the number of special variables, but fortunately an increase in the number 
of variables does not appreciably increase the computation time. Hence, 
by separable programming we have a greater control on the accuracy of 
the solution. In addition, solutions can be obtained for any value of 
required total generated power whereas this will require finer increments 
of generated power in dynamic programming. The separable program- 
ming method, of course, will require a computer code (usually the 
simplex method) for solution, while dynamic programming does not 
involve any special method. 


Example of Series Operation 


Consider the three-stage production process of Fig. 10-5. The products 
manufactured at stage 1, x9, have a market value of $10 each. These 
products can be manufactured from input products x,, using one of three 
processes. The input-output relationships of these processes are given in 
Fig. 10-6 as a function of types of process (decision) d,,d,, and d3. 
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figure 10-5. Series production process. 


100 


90 


Input x, 


i) 200 400 600 800 1,000 


figure 10-6. Jnput-output relationship of stage 1 with decision, d, 
as parameter. 


These decisions have the following cost schedule: 


Cost of processing each unit of x, with decision d, = $10 

Cost of processing each unit of x, with decision d= $5 

Cost of processing each unit of x, with decision d3= $5 = 
Cost of implementing decision d, = $500 

Cost of implementing decision d, = $700 

Cost of implementing decision d, = $600 


The input-output relationship of stage 2 is given as a function of the 
type of process (decision) in Fig. 10-7. In stage 2 two types of decisions, 
with the following cost schedule, are possible: 
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50 


40 


0 10 20 30 40 50 
Output x, 


figure 10-7. /nput-output relationship of stage 2 with decision, d, 
as parameter. 


Cost of processing each unit of x» with decision d, = $10 
Cost of processing each unit of x5 with decision d, = $5 
Cost of implementing decision d, = $600 

Cost of implementing decision d, = $500 


In stage 3 two types of decision are possible. The cost schedule of 
these decisions is as follows: 


Cost of processing each unit of x, with decision d, = $10 
Cost of processing each unit of Xp with decision d, = $20 
Cost of implementing decision d, = $500 
Cost of implementing decision d, = $100 


With the above data it is desired to obtain the optimum decisions of each 
stage for producing 100, 200, 300, ..., 1,000 units of x) for maximum 
return. 
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For the solution of the problem consider stage 1 and obtain the values 
of input x, for each value of xg and decision d,. These values are given 
in Table 10-5. Asan example of calculating the values of this table, con- 
sider output xp of 500 units and decision d). From Fig. 10-6, an output 


Table 10-5. Input and Return of Stage 1 


Decision d, Decision dy Decision dz 
Input x, /return, $ Input x, /return, $ Input x, /return, $ 
100 19/ 310 Sf 275 10/ 350* 
200 28/1,220 10/1,250 20/1,300* 
300 35/2,150 16/2,220 30/2,250* 
400 40/3,100 21/3,195 40/3,200* 
500 43/4,070 27/4,165* 50/4,150 
600 45/5,050 34/5,130* 60/5,100 
700 47,6,030 41/6,095* 70/6,050 
800 48/7,020 50/7,050* 80/7,000 
49/8,010* 62/7,990 90/7,950 
50/9,000* 80/8,900 100/8,900 


of 500 units corresponds to an input x, of 27 units using decision dy 
process. Since the selling price of each unit of x9 is $10, the total return 
of stage 1 will be 


R = 500 x 10 - (27 x 5+ 700) = $4,165 


where 5 is the cost of processing each unit x, with decision d,, and 700 
is the initial cost of implementing decision d,. The stage return and the 
number of inputs required are separated by a slash (/) in Table 10-5. 
After completing the values required in this table we choose, for each 
output x), the most profitable operating decision-highest return. These 
values are marked with an asterisk. From now on we need only concern 
ourselves with these values. 

Moving to a two-stage process consisting of stages 1 and 2, we tabulate 
the values of x, and the values of x, corresponding to the most profitable 
operation of stage 1. Columns 1, 2, and 3 of Table 10-6 represent these, 
values along with the corresponding return of stage 1. Columns 4 and 5 
give the corresponding number of input units x, obtained for the output 
units x, of stage 2 obtained from Fig. 10-7. These columns also include 
the combined two-stage return. As an example of calculations, consider 
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the calculation involved for x) of 800 and decision d, as represented in 
column 5 of Table 10-6. For an xq of 800, the corresponding x, from 
Table 10-5 is 50 with a return of $7,050. From Fig. 10-7, an x, = 50 
corresponds to an input to stage 2, x, of 50 using decision d,. The 
corresponding return of the two-stage process is calculated as follows: 


R = 7,050 - (50 x 5 + 500) = $6,300 


where 5 is the cost of processing each unit of x, and 500 is the cost of 
implementing decision d,. These costs are subtracted from the one-stage 
return of $7,050. After completing Table 10-6 the returns are optimized 
with respect to decisions and distinguished by an asterisk. 

The return of the three-stage process can be computed in a similar 
manner and a table of values obtained. As was the case with the values 
of Table 10-6, only the asterisked values of Table 10-6 will be used in 
the compilation of the three-stage process (Table 10-7). This table gives 
the most profitable two-stage values in columns | through 4 as obtained 
from Table 10-6. The three-stage returns are calculated for decisions d, 
and d, of the third stage. For example, for a value of x) = 700 
decision d, will result in the three-stage return: 


R = 5,390 - (41 x 10 + 500) = $4,480 


where 41 is the number of stage-three outputs required, 10 is the cost of 
producing each unit, and 500 is the initial cost of implementing 


Table 10-7. Three-stage Return 


Two-stage Decision d, Decision dy 
units | units a return, $ Three-stage return, $ Three-stage return, $ 

-200 -800 -500* 

200 20 | 20 700 0 200* 

300 30 | 30 1,600 800 900* 

400 40 | 40 2,500 1,600 1,600* 

500 27| 27 3,530 2,760 2,890* 

600 34 | 34 4,460 3,620 3,680* 
700 41} 41 5,390 4,480* 4,470 
800 50 | 50 6,300 5,300* 5,200 
7,265 6,275* 6,185 
8,250 7,250* 7,150 
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decision d,. The most profitable three-stage returns are marked by aster- 
isks. Thus Tables 10-5 through 10-7 give the most profitable decisions 
for producing quantities of x). The results can be combined and written 
in a single table, as in Table 10-8. This table gives the required decision 
at each stage and the resulting profit for producing various quantities of 
From this table it is seen that manufacturing 100 units of x will 
incur a loss of $500, while manufacturing 1,000 units will bring a profit 


Xo: 


of $7,250, 


PROBLEMS 


1. Find the optimum level of operating generators A, B, and C of the 
example problem for generating 60 MW by taking combinations of 
generators A and C first, and combining the results with generator 
B. 

2. Find the optimum level of operating generators A, B, and C of the 
example problem for generating 80 MW of total power. Check your 
answer with the value given in Fig. 10-3. 


Digital Computer Methods in Engineering 


3. Repeat the above problem for 70 MW of total generated power. 


4. In the three-stage production problem, assume the following cost 
figures for the first stage: 


Table 10-8. The Overall Decision Process and Return of the Three-stage 


Series Example 


Output Overall 
Stage 1 Stage 2 Stage 3 Profit, 
units | 41 dy ds dy CESS dy $ 
100 Vv Vv Vv -500 
200 Vv Vv Vv 200 
300 Vv Vv Vv 900 
400 Vv Vv Vv 1,600 
500 Vv Vv Vv 2,890 
600 v Vv Vv 3,680 
700 Vv Vv Vv 4,480 
800 Vv WE oil Pad 5,300 
900 | V Vv Vv 6,275 
1,000 | ¥ Vv Vv 7,250 
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Cost of processing each unit of x, with decision d, = $8 
Cost of processing each unit of x, with decision d= $4 
Cost of processing each unit of x, with decision d, = $2 
Cost of implementing decision d, = $500 
Cost of implementing decision d, = $500 
Cost of implementing decision ds, = $400 


and obtain the optimum production decisions as was done in the 
example problem. 

Taking the cost figures and curves of the three-stage production 
problem, calculate the most economical decisions for producing 
9's in 50, 150, 250, 350, and 450 units. 

In the three-stage production problem, drop decision d, from con- 
sideration in stage 1. With only decisions d, and dy remaining, 
calculate three-stage return for producing 100, 200, 300, and 400 
units of x9. 
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APPENDIX 
DIFFERENTIAL EQUATIONS 


DEFINITIONS 


The term ordinary differential equation is given to an equation contain- 
ing one or more ordinary derivatives or differentials; if the derivatives are 
partial derivatives, the equation is called a partial differential equation. 
For example, 


=~ 4A Bee 0 (A.1) 


is an ordinary differential equation, while the equation 


2: 12: 2 
TaN aN GV eG (A.2) 
ax? ay? dz? 
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is a partial differential equation. Note that in Eq. (A.1) the variable x is 
a function of ¢ only, while in Eq. (A.2) the variable V is a function of 
x, y, and z. 

The order of a differential equation is the order of the highest derivative 
involved in the equation. For example, Eqs. (A.1) and (A.2) are both of 
the second order. 

The general solution of an nth-order differential equation contains n 
arbitrary constants. Consider the solution of the first-order differential 
equation 


eens 2x (A.3) 
dt 


Separation of variables results in 


= dt (A.4) 
1+ 2x 


Integration gives 


3 inl PEON) et C (A.5) 


in which C is the arbitrary constant. The constant C can be evaluated 
using the initial condition. For x = 1/2 at t = 0, we get 


c =1in2 (A.6) 


1 
2 
The solution (A.5) after this substitution results in 


xa etal (A.7) 


The general solution of the second-order differential equation 


ax 10 (A.8) 


382 Digital Computer Methods in Engineering 


is given by 
x = Asinwt + B coswt (A.9) 


where A and B are the two arbitrary constants. The fact that Eq. (A.9) 
is a solution of Eq. (A.8) can be verified by obtaining the second 
derivative of x with respect to time and substituting in the differential 
equation (A.8). Given the initial conditions, the constants A and B can 
be evaluated. 

A linear differential equation is one in which the dependent variable 
and its derivatives appear only in the first power. The differential 
equations 


#y 34 oy 20 (A.10) 
dx? dx 

and 
2 
LEE eee = Le (A.11) 
dx? dx 


are linear, while the differential equations 


2 
£ , 3(#) +2y = 0 (A.12) 
dx? dx 
and 
#y 3% , 8 <0 (A.13) 
dx? dx 


are nonlinear, 

The principle of superposition of solutions applies in the case of 
linear differential equations. This principle can be stated as follows. If 
y = f(x) and y = g(x) are solutions of a linear differential equation, then 
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y = f(x) + g(x) is also a solution. It can be seen by substitution that 
y = Ae*, where A is constant, is a solution of Eq. (A.10). Again by 
substitution, it can be seen that y = Be~2*, where B is a constant, is also 
a solution of Eq. (A.10). By the superposition principle y = Ae~* + Be72* 
contains two constants A and B and will be the complete solution of the 
second-order linear differential equation (A.10). 

LINEAR EQUATIONS OF THE FIRST ORDER 


Consider the first-order differential equation 


& . py - (A.14) 


dx 


where P and Q are functions of independent variable x. We wish to 
obtain the solution y in terms of x. Equation (A.14) can be written as 


dy + Pydx = Qdx (A.15) 
Multiplying Eq. (A.15) by R, an unknown function of x, we get 
Rdz + yRPdx = RQdx (A.16) 


Comparing this equation with the identity 


Rdy + ydR = d(Ry) (A.17) 
we get 

dR = RPdx (A.18) 
and 

d(Ry) = RQdx (A.19) 


From Eq. (A.18) we obtain 


i ee = exo( fra) (A.20) 
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From Eq. (A.19) we obtain 


Ry = f ecax Bay OY 


(A.21) 


where C is an arbitrary constant. Since Eq. (A.20) gives the value of R 
as a function of x, Eq. (A.21) will give the solution y as a function of x 


and the arbitrary constant C. 


As an example of the above, consider the solution of the differential 


equation 


which results in 


R= wo ( f-2a) = exp(Inx~2) = x72 
x 


Using this in Eq. (A.21) will give 
x72y = fe + QxSdx + C 
= 7406 
or 
y = x4 4 Ox? 


where C is the arbitrary constant. 


(A.22) 


(A.23) 


(A.24) 


(A.25) 


(A.26) 
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LINEAR EQUATIONS WITH 
CONSTANT COEFFICIENTS 


Consider the linear differential equation 


ayD"y + aD" ly + + +, _,Dy + a,y = R (A.27) 


n 


where a is a constant and D is the differential operator, D = d/dx, and R 
is a function of the independent variable x. Equation (A.27) can be 
written 


f(Diy 
f(D) 


f 
Ee) 


(A.28) 


f 
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SOLUTION FORR = 0 


Consider the case of Eq. (A.27) when R = 0 and n= 1. This will result 
in 


(D -1,)y = 0 (A.29) 


where r,, = @,/a,_, = 4/dg. Equation (A.29) can be written 
CA tan (A.30) 


resulting in the solution 
y = C,, exp(r,x) (A.31) 


Generalizing this, if the roots of the equation f(D) = 0 are distinct and 


equal to r,,1r9,--.,7, then the solution y becomes 


y= Get a Coen. + Ge" beet Gate (A.32) 


As an example of this consider the second-order differential equation 


ae ae (A.33) 
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The roots of 


f(D) = 20? +D-1 
(A.34) 
= (2D -I)(D+ 1) 
arer, = 1/2, ry = -1. This results in the solution 
ie Clee = Coe (A.35) 


If the roots of the equation f(D) = 0 are multiple roots with multiplicity 
n such that the differential equation becomes 


(D -1n"y = 0 (A.36) 


then the solution can be shown to be 
y = (Cy + Cox + (ep feet Cees (A.37) 


As an example of this consider the differential equation 
2 
— -6—+9y = 0 (A.38) 


resulting in an f(D) of 


De GD en9 
(D - 3 


f(D) (A.39) 


iT 


with multiple roots r, = rg = 3. The solution of Eq. (A.38) according 
to Eq. (A.37) will be 


y = (C, + Cone (A.40) 
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If the roots of the equation /(D) = 0 are complex conjugate roots 
m,= a+ jb and m, = a — jb, then the solution 


y = Cye"* + Cye™™ (A.41) 
can also be written as 
y= K,e™* sinbx + K,e™* cos bx (A.42) 


where C,, Cy, K, and Kg are arbitrary constants. 
For example, the differential equation 


2 ; 
Fy 6% . ay = 0 (A.43) 
dx? dx 


with the roots of D? + 6D + 25 = 0 being equal to~3 + 4j and-3 - 4j 
will have a solution y = Reare® cos 4x + Kent sin4x. 


Solution for R 4 0 
For R 4 Othe differential equation (A.27) can be written in the form 
f(D)y = R (A.44) 


The solution of the differential equation (A.44) in this case will be the 
sum of two solutions y, and yg where y, is the solution of 


f(D)y, = 0 (A.45) 
and yp is the solution of 

f(D) yg = R (A.46) 
In Eq. (A.45) the value of R is set equal to zero. The solution of Eq. 
(A.45), 1, is called the complementary function while yz, the solution 


of Eq. (A.46), is called a particular solution. The solution of Eq. (A.45) can 
be obtained by methods described previously. The solution of Eq. (A.46) 
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can be obtained by assuming yg to be proportional to the sum of func- 
tions appearing in R (for example, x”, e?*, singx, cosux) and their 
derivatives as will be described in the following examples. 
Example 1. Solve the differential equation 

(D? + 2D + l)y = 4e* (A.47) 
The equation corresponding to Eq. (A.45) can be written 

(D? +2D+ Dy, = 0 (A.48) 
This results in the complementary solution 


y= (C, + Coxde™ (A.49) 


The particular solution yy of Eq. (A.46) is obtained by assuming an 
exponential function of the type given in Eq. (A.47), R = 4e*, 


ye ae (A.50) 


Obtaining appropriate derivatives and substituting in Eq. (A.47), we 
obtain 


Ae* + 2Ae* + Ae* = 4e* (A.51) 
which results in A = 1. Thus the complete solution of Eq. (A.47) becomes 
Y= + Yo = (Cy +Coxde™™ + e* (A.52) 
where C, and Cy, are the two arbitrary constants of the second-order 


differential equation (A.47). 
Example 2. Solve the second-order differential equation 


2 aL ae ee ae (A.53) 


dx? dx 
The complementary solution is obtained from the equation 


(D? - D)y, = 0 (A.54) 
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as 
y = Cy + Cye* (A.55) 


Considering the right side of Eq. (A.53) the particular solution y2 will 
contain functions of the same form, 


x2, x, constant, e*, sinx (A.56) 


We note that the constant term also appears in solution y, of Eq. (A.55) 
as C,. For this reason the powers of x appearing in Eq. (A.56) should 
be multiplied by x, 


Yq = Ax® + Bx? + Cx + ++: (A.57) 


where A, B, and C are constants. The term e* of Eq. (A.56) also appears 
in the complementary solution y, of Eq. (A.55). This term should also 
be multipled by x resulting in 


Yq = Ax® + Bx® + Cx + Bxe* + +++ (A.58) 


Adding the term sinx and its derivative cosx, we can write the particular 
solution as 


Yous Ax? + Bx? + Cx + Exe* + F sinx + G cosx (A.59) 


Obtaining the derivatives of the above equation and substituting in Eq. 
(A.53), we will obtain an equation containing the constants A, B,C, ... 
and the functions appearing in Eq. (A.59). Equating the coefficients of 
the like terms, we can solve the resulting simultaneous equations for the 
values of the constants, 


A= -1,B =-1,C =-7,E =2,F =-3,G-=% (A.60) 


The complete solution of differential equation (A.53) now becomes 
y = Cy + Cge* - x9 - x? ~ 7x 4 Qxe* -Fsinx+tcosx (A.61) 


where C, and Cy are constants. 
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SYSTEM OF LINEAR EQUATIONS 


The sets of simultaneous differential equations can also be solved using 
the methods discussed above. Consider the set of differential equations 


dy dz 
ire 


axe “dx 


= 4y+1 
(A.62) 
2 = By + 2 


where y and z are functions of the independent variable x. Equation 
(A.62) can be written as 


(D - 4)y + Dz = 1 
(D - 3)y +z = x? 


(A.63) 


Multiplying the second equation by D and subtracting from the first we 
get 

(D? - 4D + 4)y = (D - 2)"y = 2x -1 (A.64) 
The complementary and particular solutions of Eq. (A.64) are as follows: 

% = (Cy + Cox) e** (A.65) 
and 
(A.66) 
This results in 

1 


y = (Cy + Coxde®* + : ae (A.67) 


Solving the second of Eqs. (A.63) for z and substituting the value of y 
and its derivative from Eq. (A.67), we get 


2 = (C, — Cy + Coxde?* + x? + ox fa (A.68) 


cs) 
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As a second example, consider the solution of the sets of equations 


= 0 
dt 
zg (A.69) 
Bel: oF ) =6 
dt? dt 
The above equations can be written 
D?x + 2Dy = 2a 
(A.70) 
2Dx - D?y = 2b 
Solving the above for x, we obtain 
(D? + 4D)x = 4b (A.71) 
The solution of Eq. (A.71) is 
x = C, + Cy sin2t + Cy cos2t + bt (A.72) 


Using the second derivative of Eq. (A.72) in the first of Eqs. (A.69) 
we get 


Dy = 2Cy sin2t + 2C3 cos2t + a (A.73) 
Integrating this we obtain 
y = -Cy cos2t + Cz sin2t + at + Cy (A.74) 


Equations (A.72) and (A.74) with the four constants C,, Cg, C3, and C4 
constitute the solution of the two second-order differential equations 
(A.69). 
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PROBLEMS 


1. Show that y = A sinwx + B coswx is the solution of the differential 
equation 


2. Show that y = A sin(wx + B) and y = C cos(wx + D) are the solu- 
tions of the differential equation 


3. Solve the differential equation 
x dy + 8y a2 + 1 
dx 


and check your solution by substituting in the differential equation. 
4. Solve the differential equation 


(D* - 6D? + Dy = 0 
5. Obtain the solution of differential equation 


dy 
—+4—=0 
a3 dx 


Answer: y = C, + Cy cos2x + Cy sin2x 
6. Obtain the solution of the differential equation 
(D? + 7D + 12)y = 2e%* 


Answer: y = C,e3* + Cye4* 4 Lede 


10. 


11. 


12. 


Differential Equations 


Solve the differential equation 

(D? + D.- 12)y = xe* 

Answer: y = C,e** + Cye4* - 1 xex _ 3 ex 
Solve the differential equation 


(D? — 2D + l)y = 2xe* 


Hint: The particular solution contains the terms x%e* and x%e*. 


Solve the differential equation 


(D* + 4D3)y = x2 


xe xe 


Answer, y= Cre ** + Cy + Cox + Cyx® + — - 
iis che RS ory 

Solve the differential equation 

(D2? + 4)y = sin2x 


Answer: y = C, sin2x + Cy cos2x - i cos 2x 


Solve the set of simultaneous differential equations 


dy 

— —-6% + 2y = 0 
pee 

dy 

— -Ty+3x =0 
dt 


ah 


xo 


240 
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Adam's method, 233 
Addition of matrices, 14 
Adjoint matrix, 20 
Alternating direction method, 317 
Arbitrary constants of differential equation, 
381 
Augmented matrix inversion, 27 
computer program, 32 


Backward difference operator, 185 
Backward differences in boundary value 
problems, 284 
Bairstow’s method (Lin-Bairstow’s method), 
134 
computer program for, 138 
Basic solution in linear programming, 343 
Bocher, M., 56 
Boundary conditions, mixed, 280 
Boundary value differential equation, 225, 
270, 286 
forward and backward differences in, 284 
iterative methods, 271 
simultaneous, 286 
Boundary value problems: 
derivatives in, 280 
two-point, 270 


Calculus of finite differences, 183 
Cartesian coordinates, 307 
Central difference operator, 186 
Characteristic equation, 51, 54 
by method of Danilevsky, 88 
Characteristic matrix, 51, 54 
Characteristic polynomial, 51, 54 
coefficients of, 55 
Newton’s formula of, 55 
by method of Krylov, 57 
properties of, 54 
Characteristic values, 49 
computer program for, 77 
smallest, 81 
in vibration problems, 98 
Characteristic vectors, 49 
computer program for, 77 
equation of, 49 
iterative methods, 73 
by method of Danilevsky, 96 
by orthogonality, 83 
in vibration problems, 98 
Chio’s method, 6 
Coefficients of characteristic polynomials, 55 
Cofactors of determinants, 3 
Column matrix, 13 
Complementary solution: 
of difference-differential equation, 291 
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396 Index 
Complementary solution (Cont'd) 
of differential equation, 387 
Complex conjugate roots by Graffe’s method, 
131 
Computer program: 
for Danilevsky’s method, 94 
for Fourier transform, 172 
for least-squares method, 150 
for Lin-Bairstow’s method, 138 
for matrix iteration, 77 
for roots of polynomials, 138 
Computer simulation, nonlinearities in, 260 
Conformable matrices, 16 
Constraints, 340 
Continued product of matrices, 17 
Continuing solution of differential equations, 
225 
Cooley-Tukey method, 175 
Corrector equation, 227 
Cramer’s rule, 10 


Danilevsky’s method: 
for characteristic equation, 88 
for characteristic vector, 96 
computer program for, 94 
for eigenvectors, 96 
Derivative: 
in boundary conditions, 280 
of polynomial by synthetic division, 121 
of tabulated function, 200 
Derivatives, partial, 380 
Determinant evaluation, FORTRAN program 
for, 8 
Determinants, 1 
cofactors of, 3 
evaluation of, 4 
minors of, 2 
of nth order, 2 
properties of, 4 
transpose of, 2 
Deviation, standard, 264 
Diagonal form of symmetric matrices, 108 
Diagonal matrix, 107 
Difference-differential equations: 
complementary solution of, 291 
particular solution of, 293 
Difference-differential operators, 187 
Difference operators, 185 
backward, 185 
central, 186 
forward, 186 


Difference operators (Cont'd) 
Laplace’s, 309 
in partial differential equations, 307 
Differential equations, 214, 380 
arbitrary constants of, 381 
boundary value (see Boundary value dif- 
ferential equation) 
complementary solution of, 387 
with constant coefficients, 385 
continuing solution of, 225 
corrector equation for, 227 
errors in numerical solution, 294 
of first order, 237, 383 
initial values of, 225 
linear, 382 
by method of Runge, Heun, and Kutta, 
245 
methods for continuing solution, 225 
nonlinear, 382 
nth order of, reduction of, 218 
numerical solution of (see Numerical 
solution of differential equations) 
order of, 381 
ordinary, 214, 380 
partial, 305, 380 
difference operators in, 307 
elliptic, 306 
hyperbolic, 306 
Laplace’s, 306 
parabolic, 306 
quasi-linear, 306 
particular solution of, 387 
second-order (see Second-order differen- 
tial equations) 
simultaneous, 237, 252, 390 
solution: 
by Adam’s method, 233 
by direct substitution, 253 
error analysis in, 294 
by Euler method, 222 
by Fox-Euler method, 235 
by Milne’s method, 226 
by Taylor’s method, 218 
stability of numerical solution, 231, 290 
starting of solution, 215 
Discrete Fourier transform, 158 
Displacement method, successive, 316 
Division: 
of matrices, 20 
synthetic, 120 
Dynamic programming, 360 
optimality principle, 362 
parallel operation in, 364 


Dynamic programming (Cont'd) 
recursion equation of, 363 
series operation in, 372 


Eigenvalues: 
of inverse matrices, 68 
of matrix, 51 
multiple, 69 
of orthogonal matrices, 66 
repeated, 69 
of skew-symmetric matrices, 66 
of symmetric matrices, 65 
of transposed matrices, 103 
Eigenvectors, 49, 52 
determination of, 62 
equation of, 49 
of inverse matrices, 68 
by method of Danilevsky, 96 
properties of, 64 
Elliptic partial differential equation, 306 
Equality of matrices, 14 
Equations: 
characteristic (see Characteristic equation) 
corrector, 227 
differential (see Differential equations) 
of eigenvectors, 49 
heat (see Heat equation) 
homogeneous sets of, 11 
linear algebraic, 1 
linear differential, 382 
linear sets of, 11 
linear simultaneous, 12 
Poisson’s, 315 
polynomial, Newton’s method for, 125 
restraint, 340 
roots, 117 
simultaneous (see Simultaneous equations) 
transcendental, 125 
wave, 306, 330 
Error analysis in solution of differential 
equations, 294 
Euler method, 222, 235 
Evaluating determinants, 4 
Explicit methods, definition of, 320 


Fast Fourier transform, 157 
Cooley-Tukey method, 175 

Feasible solutions, 345 

Finite differences, calculus of, 183 
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First-order differential equations, 237, 383 
FORTRAN program: 
for determinant evaluation, 8 
for matrix inversion, 32 
Forward difference operator, 186 
Forward differences in boundary value prob- 
lems, 284 
Fourier transform, 157 
computer program, 172 
Cooley-Tukey method, 175 
discrete, 158 
fast (see Fast Fourier transform) 
Fox-Euler method, 235 
Frequency analysis, 157 
Functional (objective form), 340 
Functions, tabulated (see Tabulated func- 
tions) 


Gauss’ method, 21 
Gauss-Seidel method: 
for Laplace’s equation, 316 
for simultaneous equations, 37 
Graffe’s method, 127 
for complex conjugate roots, 131 
Gram-Schmidt orthogonalization process, 
104 
Gregory-Newton interpolation formula, 208 


Heat equation, 306, 320 
general solution of, 325 
two-dimensional, 329 
Heun, Runge, and Kutta, integration formu- 
las of, 245 
Heun’s formula, 247 
Heun’s method, 245 
Homogeneous sets of equations, 11 
Horner's method, 124 
Hyperbolic partial differential equation, 306 


Implicit methods: 

definition of, 319 

for Laplace’s equation, 317 
Infeasible solution, 340 
Initial value problems, 225 
Integral of tabulated functions, 201 
Integration formulas: 

for polynomials, 205 

of Runge, Heun, and Kutta, 245 

of Simpson, 207 

trapezoidal, 202, 206 
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Index 


Interpolation, inverse, 124, 145 
Interpolation formulas, 142, 208 
Gregory-Newton, 208 
Lagrange’s, 117, 142 
Newton’s, 208 
Sterling’s, 209 
Inverse interpolation, 124, 145 
Inverse matrix, 20 
eigenvalues of, 68 
eigenvectors of, 68 
by partitioning, 34 
Iterative methods: 
in boundary value problems, 271 
for characteristic vectors, 73 
for Laplace’s equation, 311 


Krylov’s method for characteristic poly- 
nomial, 57 

Kutta, Runge, and Heun, integration formu- 
las of, 245 

Kutta-Runge method, 252 

Kutta’s fourth-order formula, 248 

Kutta’s method, 245 

Kutta-Simpson three-eighths rule, 248 

Kutta’s third-order rule, 248 


Lagrange’s interpolation formula, 117, 142 
Laplace’s equation: 
Gauss-Seidel method for, 316 
general solution, 315 
implicit methods for, 317 
iterative methods, 311 
Liebmann’s method, 311 
relaxation factor in, 316 
solution of, 308 
Laplace’s operator, 309 
Laplace’s partial differential equation, 306 
Least-squares methods, 117, 145 
computer program for, 150 
Liebmann’s solution of Laplace’s equation, 
311 
Lin-Bairstow’s method, 134 
computer program for, 138 
Linear algebraic equations, 1 
Linear differential equations, 382 
Linear partial differential equations, 306 
Linear programming, 338 
basic solution of, 343 


Linear sets of equations, 11 
Linear simultaneous equations, 12 


Matrices: 
addition of, 14 
adjoint, 20 
algebra, 12 
calculation of Fourier transform by, 162 
characteristic, 51, 54 
column, 13 
conformable, 16 
continued products of, 17 
definitions, 13 
diagonal, 107 
division of, 20 
eigenvalues of, 51 
equality of, 14 
FORTRAN program for, 32 
inverse (see Inverse matrix) 
inversion of, 27 
iteration of, computer program for, 77 
iteration method for characteristic vectors, 
73 
modal, 106 
multiplication of, 15, 18 
properties of, 17 
notation, 13 
orthogonal, eigenvalues of, 66 
partitioned, 18 
postmultiplication, 17 
premultiplication, 17 
tow, 13 
skew-symmetric, eigenvalues of, 66 
spectral, 106 
square, 13 
symmetric (see Symmetric matrices) 
transposed, 13 
eigenvalues and eigenvectors of, 103 
multiplication of, 18 
reversal law of, 18 
unit, 15 
zero, 14 
Mean values, 264 
Method of least squares (see Least-squares 
method) 
Milne’s method, 226, 239 
predictor-corrector formula, 227 
Minors of determinants, 2 
Mixed boundary conditions, 280 
Modal matrix, 106 
Monte Carlo methods, 264 
Multiple eigenvalues, 69 


Multiplication of matrices, 15, 18 
properties of, 17 


Navigation problem, proportional, 254 
Newton’s formula for coefficient of char- 
acteristic polynomials, 55 
Newton’s interpolation formula, 208 
Newton’s method for polynomial equations, 
125 
Nonlinear differential equations, 382 
Nonlinearities, simulation of, 260 
Nonoscillatory solution, 299 
Noumerov’s method, 241 
Numerical solution of differential equations, 
214 
errors in, 294 
stability of, 231, 290 


Objective form (functional), 340 
Operators: 
difference (see Difference operators) 
difference-differential, 187 
Optimality principle, 362 
Order of differential equation, 381 
Ordinary differential equation, 214, 380 
Orthogonal matrix, eigenvalues of, 66 
Orthogonality relations in obtaining charac- 
teristic vectors, 83 
Orthogonalization process, Gram-Schmidt, 
104 
Overrelaxation methods, 316 


Parabolic partial differential equations, 306 
Parallel operation in dynamic programming, 
364 
Partial derivative, 380 
Partial difference operators, 307 
Partial differential equation (see Differential 
equations, partial) 
Particular solution: 
of difference-differential equation, 293 
of differential equation, 387 
Partitioned matrices, 18 
inverse matrix by, 34 
Poisson's equation, 315 
Polynomial equation, Newton’s method for, 
125 
Polynomials: 
characteristic (see Characteristic polyno- 
mial) 


399 


Polynomials (Cont'd) 

derivatives of by synthetic division, 121 

integration formulas for, 205 

quadratic factors of, 134 

roots of (see Roots, of polynomials) 
Postmultiplication of matrices, 17 
Predictor formula, 227 
Premultiplication of matrices, 17 
Programming, dynamic (see Dynamic pro- 

ramming) 

separable, 346, 338, 370 
Properties: 

of determinants, 4, 21 

of matrix multiplication, 17 
Proportional navigation problem, 254 


Quadratic factors of polynomials, 134 
Quadratic forms, 49 
Quasi-linear partial differential equation, 306 


Random inputs, simulation of, 260 
Random numbers, generation of, 262 
Recursion equation of dynamic programming, 
363 
Reduction of nth-order differential equation, 
218 
Relaxation factor in Laplace’s equation, 316 
Repeated eigenvalues, 69 
Restraint equations, 340 
Reversal law of transposed matrices, 18 
Roots: 
of equations, 117 
of polynomials, 118 
computer program for, 138 
Graffe’s method, 127 
Horner’s method, 124 
Lin-Bairstow’s method, 134 
Newton’s method, 125 
Row matrix, 13 
Runge, Heun, and Kutta, method of, 245 
Runge-Kutta method, 252 


Second-order differential equations: 
Milne’s method, 239 
Noumerov's method, 241 
Separable programming, 346, 338, 370 
Series operation in dynamic programming, 
372 
Similarity transformation, 67 
Simpson’s integration formulas, 207 
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Simpson’s one-third rule, 203, 207 
Simpson’s three-eighths rule, 248 
Simulation: 
of nonlinearities, 260 
of randon inputs, 260 
Simultaneous boundary value differential 
equations, 286 
Simultaneous differential equations, 237, 
252, 390° 
Simultaneous equations, 1 
in boundary value problems, 272, 276 
Gauss’ method for, 21 
Gauss-Seidel method for, 37 
by matrix inversion, 26 
triangulation method for, 23 
tridiagonal, 318 
Skew-symmetric matrix, eigenvalues of, 66 
Slack variables, 341 
Smallest characteristic value, 81 
Spectral matrix, 106 
Square matrix, 13 
Stability of numerical solutions, 231, 290 
Standard deviation, 264 
Starting solution of differential equations, 
215 
Sterling’s interpolation formula, 209 
Successive displacement method, 316 
Successive-overrelaxation methods, 316 
Superposition principle, 382 
Symmetric matrices, 64 
diagonal form of, 108 
eigenvalues of, 65 


Synthetic division, 120 
derivative of polynomial by, 121 


Tabulated functions: 
derivatives of, 200 
integral of, 201 
Taylor series, 188 
Taylor’s method, 215, 218 
Thomas’ method, 318 
Time-frequency analysis, 157 
Transcendental equations, 125 
Transposed matrices, 13 
eigenvalues and eigenvectors of, 103 
multiplication of, 18 
Trapezoidal rule of integration, 202, 206 
Tridiagonal simultaneous equations, 318 
Two-dimensional heat equation, 329 
Two-point boundary value problem, 270 


Unit matrix, 15 


Vectors: 
characteristic (see Characteristic vectors) 
Vibration problems, 98 


Wave equation, 306, 330 


Zero matrix, 14 


